Grouping Nodes with known Value Differences: A lossless UCT-based Abstraction Algorithm
Authors: Robin Schmöcker
Alexander Dockhorn
Bodo Rosenhahn
Venue: ICLR 2026 Poster (2026)
Keywords: Artificial intelligence
Sequential decision-making
Abstractions
MCTS
Primary area: optimization
TL;DR: A method to detect lossless state-action pair abstractions to enhance MCTS by grouping state-action pairs whose Q* differences are known.

Abstract: A core challenge of Monte Carlo Tree Search (MCTS) is its sample efficiency, which can be addressed by building and using state and/or state-action pair abstractions in parallel to the tree search, such that information can be shared among nodes of the same layer. On the Go Abstractions in Upper Confidence bounds applied to Trees (OGA-UCT) is the state-of-the-art MCTS abstraction algorithm for deterministic environments that builds its abstraction using the Abstractions of State-Action Pairs (ASAP) framework, which aims to detect states and state-action pairs with the same value under optimal play by analysing the search graph. ASAP, however, requires two state-action pairs to have the same immediate reward, which is a rigid condition that limits the number of abstractions that can be found and thereby the sample efficiency. In this paper, we break with the paradigm of grouping value-equivalent states or state-action pairs and instead group states and state-action pairs with possibly different values as long as the difference between their values can be inferred. We call this abstraction framework Known Value Difference Abstractions (KVDA), which infers the value differences by analysis of the immediate rewards and modifies OGA-UCT to use this framework instead. The modification is called KVDA-UCT, which detects significantly more abstractions than OGA-UCT, introduces no additional parameter, and outperforms OGA-UCT on a variety of deterministic environments and parameter settings.

Published as a conference paper at ICLR 2026
GROUPING NODES WITH KNOWN VALUE DIFFER- ENCES: A LOSSLESS UCT-BASED ABSTRACTION AL- GORITHM
Robin Schmöcker
Alexander Dockhorn
Institute for Information Processing
SDU Metaverse Lab University of Southern Denmark Odense, Denmark a adoc@mmmi. sdu..d dk
Leibniz University Hannover
Hannover, Germany
schmoecker@tnt. uni muni-hannover.de
Bodo Rosenhahn
Institute for Information Processing
Leibniz University Hannover
Hannover, Germany
rosenhahn@tnt..u un hannover.de
ABSTRACT
A core challenge of Monte Carlo Tree Search (MCTS) is its sample efficiency. which can be improved by grouping state-action pairs and using their aggregate statistics instead of single-node statistics. the Go Abstractions Upper Con- fidence bounds applied to Trees (OGA-UCT) is the state-of-the-art MCTS ab- straction algorithm for deterministic environments that builds its abstraction using the Abstractions of State-Action Pairs (ASAP) framework, which aims to detect states and state-action pairs with the same value under optimal play by analysing the search graph. ASAP, however, requires two state sate-action pairs to have the same immediate reward, which is a rigid condition that limits the number abstractions that can be found and thereby the sample efficiency. In this paper, break with the paradigm of grouping value-equivalent states or state-action pairs and instead group states and state-action pairs with possibly different values long as the dif- ference between their values can be inferred. We call this abstraction framework Known Value Difference Abstractions (KVDA), which infers the value differences by analysis of the immediate rewards and modifies OGA-UCT to use this frame- work instead. The modification is called KVDA-UCT, which detects significantly more abstractions than OGA-UCT, introduces no additional parameter, and out- performs OGA-UCT on a variety of deterministic environments and parameter settings.
INTRODUCTION
Research into non-learning-based decision-making algorithms such as Monte Carlo Tree Search (MCTS) (Browne et al. 2012 Kocsis & Szepesvári] 2006) is an active field. On the one hand MCTS can be used for applications where a general on-the-fly fly applicable decision-maker is needed such Game Studios which rarely use Machine Learning (MI.) based Al as they would have to retrained whenever the game rules are modified (e.g. during development or patches). And on the other hand, though not the scope of this paper, foundational work in MCTS might potentially translate to improvements of ML algorithms such as Alpha Zero (Silver et al. (017)) that built MCTS.
algorithm
One way to improve MCTS is to reduce the search space by grouping states and actions in the cur- rent MCTS search tree to enable intra-layer information flow (Jang et al.) 2014 Anand et al. 2013 2016) by averaging the visits and returns all abstract action nodes in the same abstract node used for the Upper Confidence Bounds (UCB) formula in the tree policy, which increases the sample

---

Published as a conference paper at ICLR 2026
efficiency. One key strength of one of the state-of-the-ang abstraction algorithms On the Go Abstrac- tions in Upper Confidence bounds applied to Trees (OGA-UCT) (Anand et al. 2016) is its exactness in the sense that if OGA-UCT groups two state-action pairs in search tree where all possible suc- cessors of each state-action pair have heen sampled, only state-action pairs are grouped that have the same Q" value, i.e., they have the same value under subsequent optimal play. This exactness con- dition, however, comes at the cost that state or state-action pairs that only differ slightly in their Q" value camnot be detected. This issue was slightly alleviated with the introduction of (Ear &)-OGA (Schmöcker et al.) 2025dj, which is equivalent to (Ea, 0)-OGA in deterministic environments that allows for small errors in the immediate reward or transition function when building an abstraction. However, this also brings two downsides with it. Firstly, it introduces two parametens, which makes tuning harder, and secondly, grouping non-value equivalent state-action pairs might even be harmful to the performance, as they can make convergence to the optimal action impossible.
In this work we propose Known Value Differences Abstractions UCT (KVDA-UCT), that relaxes the strict abstraction conditions of OGA-UCT to detect almost many abstractions as (Ea. 0)-OGA in deterministic environments but without losing the exactness condition. The novel idea that makes this possible is to deliberately group states state-action pairs that do not have the same value if we know the difference between their values which is inferred by analysis of the search tree's ' immediate rewards. When the abstractions are used, instead of averaging state-action pair values directly, their aifference-accountee values are averaged. The contributions of this paper can be summarized as follows:
1. We introduce the SetwnnValle-Diffeeee stractions of State-Action Pairs (ASAP) framework used by OGA-UCT, Fig. D is an example of a simple state-transition graph in which KVDA finds three non-trivial abstractions, while ASAP would detect none.
(KVDA) abstraction framework that extends the Ab-
2. We propose and empirically evaluate KVDA-UCT, modification of OGA-UCT that introduces no parameters and uses and builds KVDA abstractions. The introduction of KVDA-UCT aims to address the abstraction literature gap for deterministic settings since existing modifications of OGA- UCT have only focused on stochastic settings (Anand et al.) 2016 Xu et al.) 2023) [Schmöcker al. 2025d c). We show that KVDA-UCT outperforms OGA-UCT most of the here-considered deterministic environments. We also compare KVDA-UCT with (ea, 0)-OGA and show that either performs equally well or better than parameter-optimizee (Ea, 0)-OGA agent.
3. Though not the main focus of this paper, we also consider the stochastic setting where we gener- alize KVDA-UCT to KVDA which allows for errors in the transition function when building the abstraction. Furthermore, we compare it to (e,, Ea) J-GA and show that, unlike in the deterministic setting, KVDA rarely performs better than (5 -OAA in stochastic environments. Still, there some environments in which e,- VDA achieves the best performances, which allows one to view as additional tool improve the parameterroppimmized performance.
The paper is structured as follows. In Section 2 the theoretical groundwork for automatic state and state-action pair abstractions is laid. In particular, we define OGA-UCT and (ea, & Then, in Section 3 our novel KVDA framework and both KVDA-UCT and e,-VVD are introduced. Afterwards, in Section
4 the experimental setup defined, then in Section [5] the experimental results using this setup are shown and discussed. The paper is concluded by discussion of the limitations of KVDA-UCT and avenues for future work in Section 6
2 FOUNDATIONS OF AUTOMATIC ABSTRACTIONS
In this section, we will be laying the theoretical groundwork for this paper as well as introducing related work. For comprehensive overview of non-learning-bee based abstractions, refer to the survey paper by Schmöcker and Dockhorn (Schmöcker & Dockhorn 2025).
Problem model and optimization objectíve: For our purposes, finite Markov Decision Processes Sutton & Barto 2018) are used as the model for sequential, perfect-informatie decision-making tasks. A(X) ) denotes the probability simplex of a finite, non-empty set X and the power of X is
denoted by P(X).
Defimition: An MDP is 6-cuple (S, no:77 T, A, P, R) where the components follows:
2

---

Published as a conference paper at ICLR 2026
r=1
r=)
3
r= r=1
Figure 1: An example of an MDP state-transition graph where the state-of-the-ar abstraction frame- work ASAP (Anand et al. 2013) would detect no abstractions while our method Known-Value- iifference-Abstracttion
(KVDA) detects three non-trivial abstractions. In this example, circles represent states, arrows represent deterministie mate-transitions and arrow annotations denote the immediate transition reward. All actions or states that are intersected by a red ellipse will ab- stracted by KVDA.
S * 9 is the finite set of states.
E A(S) is the probability distribution for the initial state.
0 + T S S the (possibly empty) set terminal states.
A: (S T") + A maps each state the available actions 0 + A(s) E A state where
||| < co.
P: (S Т) A +
A(S) the stochastic transition function where Pss'| s, a) is used to denote the probability of transitioning from s E (S (T) to s' E S after taking action
a E (s)) in s.
R: (S T) x A R is the reward function.
Let M = (S. No: T, A. P. R) be an MDP. define P: {(s E (5 7), E A())) as the set of all legal state-action pairs. The objective is to find mapping (i.e. agent) S - A(A) such that π maximizes the expected episode's s return where the (discounted) return of an episode so; Go, Po,.... . Sn, On Fun 8n+1 with 8a+1 E T is given by 7 +... Tm
bstracciins of State-Action Pairs (ASAP): For MCTS- based abstraction research, the goal has been detect state-action pairs with the same Q" value (the value under subsequent optimal play) in the search graph to increase sample efficiency by an intra-layer information flow (Jiang et al. 2014 Anand al. 2015 2016). In general, by abstractions of either the states of state-action-pairs refer to equivalence relations over the state set S or state-action pair set P. The equivalence classes are abstract states or state-action pairs.
The current state of the art is the Abstraction of State-Action Pairs in UCT (ASAP) abstraction framework (Anand et al. 2015) that proposes rules detect value-equivalent states and state-action pairs given an MDP transition graph and applies it to the current MCTS search graph (for details on MCTS, see Section A.8). The core idea of ASAP is alternatingly construct a state abstraction given a state-action pair abstraction and a state-action pair abstraction given state abstraction.
Assume one given state abstraction E' C S x The corresponding ASAP state-action pair abstraction H. C P x P is defined as grouping those state-action pairs with the same immediate reward and equal abstract successor distribution. Concretely, any state-action-pair (s1, a1), (82, a2) is equivalent i.e. ((s1, a1); (82,02)) H if and only if
F RR(11,
R(s2, a2) = 0
==
Σ TEX
Σ S'En
P(s $1, 01)
P(s' sy (z) 0,
where X' are the equivalence classes of E'. And given a state-action pair abstraction H' C x P, the corresponding ASAP state abstraction & groups all states whose actions can be mapped each
3

---

Published as a conference paper at ICLR 2026
other, concretely:
(s), 92) E =
Ya1 E A(1)) Bay A(2)) ((51, 01), (52, 02)) H4+1
Vaz E A Ea1 E A(s)): (111, (82, 02)) E Hs+1-
To obtain the ASAP abstraction for a given MDP, these two constructing steps are repeated altemat-- ingly until convergence. Note that the ASAP abstraction differs from MDP bisimulations as well as MDP homomorphisms (Ravindran & Barto 2004) because state-action pairs can abstracted even their parent states are not abstracted.
OGA-UCT: Anand al. (2016) proposed OGA-UCT, which builds an ASAP-like abstraction in parallel to running MCTS. When building the abstraction OGA starts with the initial state abstrac- tion that groups all terminal states of the same layer and puts the remaining states in their own sin- gleton equivalence class. Furthermore, when building the ASAP abstraction on the current search graph, OGA ignores non-yet-see -on-yel-s successors of acttenacioon pairs that appear in Equation [ To make the frequent recomputation of the ASAP abstraction feasible, OGA keeps track of recency counter for each Q-node and once it surpasses a certain threshold, recomputes its abstraction. If the abstraction changed, the parent states are recomputed too (and possibly their Q-node parents if their abstraction changed). By only locally checking for errors in the abstraction, OGA is able to keep track of an ASAP-like abstraction that is always close the true ASAP abstraction of the current search tree.
The only MCTS component that OGA-UCT affects is the tree policy which is enhanced by using the aggregate returns and visits of Q-node's abstract node to enhance the UCB value.
(E., -GGAAA The ASAP framework groups only value equivalent states and state-action pairs. This condition can be relaxed like already done by predecessor of OGA-UCT, called AS-UCT (Jiang et al. 2014), by allowing the rewards in Equation D to differ by some threshold ea > 0 and the transition error F; of Equation [ to lie in the interval [0, ai] where 0 S s S is another parameter which does not have any effect in deterministic environments. Since in general, positive threshold values do not induce an equivalence relation over state-action pairs, OGA-UCT has to be slightly modified to accommodate these approximate abstractions. This has been done by Schmöcker and Dockhorn (Schmöcker et al. 2025d) who introduced (e., -)-AAA
Auther automatic abstraction algorithm: The ASAP framework is the direct successor of Ab- straction of States (AS) by Diang et al. (0014) that abstracts states if and only if their actions are pair- wise similar in the sense that Equation 2 only has to be approximately satisfied described in the (Ea, G-G section above. While this paper focuses on deterministic domains, (en, -)-OGA (and related algorithms) have been further improved for stochastic settings by defining intra-abstraction policies (Schmöcker et al., (025c)), or dynamically abandoning the abstraction (Xu et al.) 2023 Schmöcker et al. 2025d).
Yet another paradigm is demonstrated by Hostetler et al. 2015) and Schmöcker et al. (025a) who
optimistically construct abstractions by starting with very coarse abstraction (e.g. grouping every- thing together) and then they refine this abstraction, for example by repeatedly splitting large node groups in half (Hostetler et al. 2015)..
Research effort has also been dedicated towards automatic abstractions of the transition function, which on an abstract level can be described as pruning certain successors from the transition function (Sokota et al. 2021
Yoon et al. 2008 2007 Saisubramanian al. 2017).
3 METHOD
This section introduces our novel KVDA-UCT algorithm which is designed to detect strictly more abstractions than OGA-UCT which allows one to compute more accurate UCB values during both the decision and tree policy which in turn leads to performance improvements. First, we will provide an example of a concrete search graph in which ASAP misses abstractions that the Known Value Differences Abstractions (KVDA) framework would find which will be introduced after the search graph example. At the end of this section, we will be describing how K KVDA is integrated into UCT to yield KVDA-UCT.
4

---

Published as a conference paper at ICLR 2026
3.1 WHICH ABSTRACTIONS ASAP MISSES
Consider the state-transition graph that is illustrated in Fig. T that consists of four states and four actions. The ASAP framework would not detect any equivalences. In fact, any framework that aims at finding value-equivalent states state-action pairs would at most be able to detect that the two root actions are value-equivalent. However, by analysing the state graph, one could derive that the Q" values of the action of node 2 differs only by 1 from the Q" value of the action of node 3. This holds true even if the state-graph would extend past node 4. Consequently, the V values of nodes 2 and 3 differ only by This in turn implies that the two actions of node must have the same Q* value.
The analysis of this example be further generalized which is the core idea of Known- Value- Diteerence-Absracctinnn (KVDA) framework which is to abstract/group states and state-action pairs one can derive their value differences. Intuitively, KVDA abstractions are identical to ASAP abstractions except that immediate reward errors are ignored and accounted for by tracking them as value differences. Later, when using these abstractions enhance MCTS, the differences only have to be subtracted when aggregating values. When viewed from the lenses of the KVDA framework, ASAP only groups states or state-action pairs with a value difference of O, i.e. detects only true equivalences.
While this was only a toy example of illustration purposes, the KVDA framework does also discover more abstractions in practice will later be shown experimentally. A concrete example of an MDP in which KVDA detects strictly more abstractions than ASAP is Game of Life (see Section A.9) In this environment, the immediate reward is also equal to the number of living cells. Therefore, state-action pairs from states with different number of living cells cannot be ASAP-abstracted even though they might have the successor(s). In contrast, KVDA is able discover such abstractions.
Next, we will formalize the KVDA framework which both in theory and in our empirical evaluations (see Tab. more details given in the experimental section) detects strictly more abstractions.
3.2 THE KNOWN-
FRAMEWORK
Our method, the Known- Difference-Abstraccee (KVDA) extends the ASAP definition by additionally grouping states or state-action-paiie whose V or Q" difference is known. While ASAP iteratively builds abstractions on abstractions, KVDA bootstraps of abstraction, difference- function pair. More concretely, given a state abstraction E' (or a state-action pair abstraction H') and a difference function d²: S S IR (or d== Px P R), both state-action-pair abstraction H (or a state abstraction E) is produced as well as state-action pair difference function d, P x P + IR (or a state difference function de: S x S + RR). Both dg and de will be constructed such that
d,(p P2) = d3 = Q" (pz) (p1) ds(s1, 82)
(3)
-ss) Vs)
for any pair of states or state-action pairs in the same equivalence class (i.e., in the same abstract state). Even though in the experimental section, will mostly consider deterministic environ- ments, the now-to-be-descrrbeed KVDA framework will be applicable to any MDP, including stochastic ones. Next, starting with the base case, we will formalize how KVDA abstractions are built.
The base case: Like ASAP, KVDA groups all terminal states into the same initial abstract node, the remaining states are singleton abstract nodes. The difference function between all nodes in the terminal abstract node is initialized with 0.
State-action- abstractions: Using the same notation as in Section 2 let E' C S S be a state abstraction and d': S x S + R be a difference function. The corresponding KVDA state- action pair abstraction H C P x P is defined as follows: Any state-action pair (1), (82, a2) is equivalent i.e. (($1, 01), (82, d2)) E H if and only if
Σ TEX
Σ S'
P(5'( (1, 01) P(s'l 82, 02) = 0.
(4)
5

---

Published as a conference paper at ICLR 2026
where X' are the equivalence classes of E'. Note that this is almost identical to the ASAP definition except that the immediate rewards do not have to coincide. The difference function da: P P for two (P1.P2)= H is given by
ds(p1,P2) -R(p2) R((1))
+ +Σ> E
(P(s' pL) - P(s" p2)) (5(5,,52)
(5)
mEX mEX d'En
where for each x. E X, 85 is an arbitrarily chosen but fixed representative of X. We will later see that the value of d, is independent of this choice.
State abstractions: Given state-action pair abstraction H' P x P and state-action pair difference function da: P x P + R the corresponding KVDA state abstraction & C S x S groups all states whose actions can mapped to each other, and whose mappings all have the same value difference:
(81,82) E + 3E E :
Va1 E A(s1) E A(s2) ((s1.01), (s2, E H'
A Ad(((s1, 01), (82, 02)) d Va, E A(2)) 3a, A(s)) (()),))) (22, H' A da(((2, a2), (s). = -d.
(6)
The difference function (s $2) is defined as the value d in the equation above.
Theoretical guarantees:
Convergence: Given MDP, the above-described construction steps be repeated until conver- gence, which is guaranteed as in our MDP definition, there are finitely many states and state-action pairs and each construction step either leaves the abstraction unchanged reduces the number of equivalence classes. The abstraction that one obtains at convergence is called the KVDA abstraction of MDP.
Soundness of d, and dz: Both da and dg at convergence are equal to the differences of their arguments Q" or V* values formulated in Eq. 3 This proven the supplementary materials in Section A.2
3.3 KVDA-UCT AND -VDA
In this section, we will describe how the KVDA abstraction framework is integrated into MCTS, which is fully analogous how O)GAA (which itself equivalent OGA-UCT for éa & = 0) integrates the ASAP framework. The usage of the following to-be-described modi- fications to (Ea, -)-OGA is called eV-. For the case é = O. the algorithm is simply called KVDA-UCT. -VKA does not depend on Ea unlike (Ea, OGA. The detailed pseudocode for this method is provided in Pseudocode in which the differences to (Ea. -GG are shaded in blue.
1) Instead of (approximate) ASAP abstractions, (approximate) KVDA abstractions of the search tree are built. Both (Ea, G)-OGA and GVKVAA allow the transition error of Equation to be in the interval [].
2) For efficient distance function calculations, each abstract node keeps track of a representative. Value differences are only calculated with respect to the representative. An abstract node's repre- sentative is the first original node added the abstract node, and if that one removed, a random new representative is chosen.
3) Each abstract Q node's value is tracked as the value of its representative QR that encodes the state-action-pair pR E P. When another original node @ of the same abstract node representing p E P is backed up in the MCTS backup phase with value v E R. the value v + da(p, P&) is added to the abstract node's statistics. In turn, Q extracts its aggregated returns from the abstract node by subtracting d, (p, PR) from the abstract node's value. If an abstract node's 's representative changes to Oa encoding P/ P. then (pR, pk) ) is added to the statistics where is the abstract visit count.

---

Published as a conference paper at ICLR 2026
Note that in the deterministic setting where da is always computed correctly, this implies that KVDA- UCT will never visit state-action pairs as long as they are abstracted with another state-action pair with positive d, difference (i.e. when they provably not the optimal action) since these state- action pairs will have the same exploration term and Q values that differ by da. This a strength that OGA-UCT does not possess.
4) To reduce the computational load of finding a perfect match as the one required for the state abstractions in Equation [ we check for a stricter condition for states 81, 82. is first checked that within 81 (and analogously 82) all actions within the same abstract node have value difference of zero. Then, tested for all abstract Q nodes of $1 if the value difference between an arbitrarily chosen ground action of $1 and one of $2 is constant for all abstract nodes.
5) Since the KVDA abstraction also depends on the difference functions, whenever Q-node's recency counter reaches the threshold, the value difference to its representative is recalculated. A change this difference also results in a reevaluation (and subsequent recency counter reset) of the parent nodes' abstractions and difference functions.
4 EXPERIMENT SETUP
In this section, describe the general experiment setup. Any deviations from this setup will be explicitly mentioned.
For this paper, we either from the International Probabilistic Planning Conference (Gizes et al. (0144) are well-known board games, or are commonly used in the abstraction algorithm literature. Since we will compare KVDA-UCT to OGA-UCT we chose only environments with a non-constant and non-sparse re- ward function, as constant reward environments would imply that KVDA-UCT and OGA-UCT are semantically equivalent.
Problem models:
experiments variety of MDPs, all of which
All MDPs originally feature stochasticity, however, for some experiments we considered their de- terministic versions which are obtained by sampling a single successor of each state-action pair. For each episode, new successors are sampled. All experiments were run on the finite horizon versions of the considered MDPs with default horizon of 50 steps and 200 for the board games with planning horizon 50 and a discount factor 7 = 1.
The board games are transformed into MDPs by inserting deterministic One-Step-Lookaheed agent as the opponent. Furthermore, since they are all sparse-reward (i.e. either win or lose) they were also transformed into dense-reward MDPs by using heuristics. For each board game we defined a heuristic yh that assigns each state a hcuristic value for its state value. The reward of the (de- terministic) transition s, s' is then given by (s() (*()). The concrete heuristics used along with a brief description all MDPs is provided in the supplementary materials in Section A.99 The deterministic MDPs are denoted by adding the prefix d- and the stochastic ones are denoted by the prefix s-.
Parameters: Since the problem domains have vastly different reward scales, we the dynamic exploration factor Global Std (Schmöcker et al., (202cc) which has the form C o where a is the standard deviation of the Q values of all nodes in the search tree and C' E E is some fixed parame- ter. Furthermore, we always use K- = 3 as the recency counter which was proposed by Anand et al. (016))
Evaluation: Each experiment is repeated at least 2000 times and all confidence that we denote in the following are 99% confidence intervals with range x 2 2.33 times the standard error.
Normalized pairings score: We will later construct the normalized pairings score to test the gen- eralization capabilities of KVDA-UCT. This the same score as used in [Schmöcker et al. (025e).. The pairings score is constructed as follows. Let ( TT be n agents (e.g. each KVDA-UCT agent along with its parameter setting is an agent) where each agent was evaluated on m tasks (later, a task will be a given MCTS iteration budget and an environment). This induces matrix of size n x n with the entry (i.j) being equal to the number of tasks where π. achieved a higher perfor- mance than x, subtracted by the number of times it performed worse, divided by m. The normalized pairings score s; I < n is then obtained by taking the average of i-th row when excluding the i-th column.
7

---

Published as a conference paper at ICLR 2026
Reproducibility: For reproducibility, released implementation (Schmöcker 2025). Our code was compiled with g++ version 13.1.0 using the -03 flag (i.e. aggressive optimization).
5 EXPERIMENTS
This section presents the experimental results of KVDA-UCT. We considered two settings. Firstly, we measured the number of additional abstractions KVDA-UCT finds in comparison OGA-UCT. Then, we evaluated KVDA-UCT on deterministic settings in which the losslessness of the abstrac- tion is guaranteed. Lastly, we present the results for stochastic environments.
Abstractions that KYDA-UCT finds but OGA-UCT does not: Firstly, we empirically measured the number non-trivial abstractions (i.e. those that are not of size one) that KVDA-UCT, OGA- UCT, and (zo, O)-UCT find. For all deterministic environments, Tab. denotes the average ratio of non-trivial abstract state-action pairs (synonymously abstract Q nodes) to the number of total abstract Q nodes in their respective search trees. In most environments, KVDA-UCT detects more abstractions than OGA-UCT, including environments where the abstraction rate more than doubles, such SysAdmin Wildfire. Furthermore, KVDA-UCT detects roughly many abstractions as (oo, 0)-OGA in most environments, which fully ignores rewards when building abstractions. Of course, finding more abstractions is not a difficult task in of itself, since could just group every node by default. However, both ASAP and the KVDA abstractions are lossless in the deterministic setting, i.c. they group only state-action pairs with identical or known-difference Q" values. We empirically verified this for those environments which could be solved with value iteration, which are Saving, Sailing Wind, and Skills Teaching. Hence, KVDA is able to detect strictly more true equivalences than ASAP.
Table 1: The average ratio of abstract Q nodes in KVDA, OGA-UCT's and (co, O)-OGA's respective search trees after 1000 iterations with = 2 that encompass more than one original node divided by the total number of abstract Q nodes. The states in which these search trees are built are sampled from an OGA-UCT agent with = 2 and 500 iterations. This measurement excludes all size-one abstract Q nodes whose original Q node has yet reached the recency counter (see Section P required for the first abstraction update. Hence, a ratio of 1 mcans that no abstractions were found while ratio of 0 means that every state-action pair has been abstracted with another. Note that KVDA-UCT (our method) finds more abstractions than OGA-UCT in nearly every environment and general as many as (oc, -)-GGA While there are some environments where there no gain, such as Constrictor, there are other environments such as Wildfire SysAdmin, where the abstraction rate more than doubles.
KVDA-UCT
OGA-
(oo,0)
Domain
(nurs)
UCT
OGA
d-Academic Advising
0.69
0.73
0.68
d-Conneut4
0.80
0.91
0,80
d-Constrictor
0.98
0.97
0.97
d-Cooperative Recon
0.44
0.56
0.48
d-Earth Observation
0.65
0.99
0.68
ElEeevatoss
0.28
0.32
0.29
d-Game Life
0.54
0.56
0.55
Manufactuurr
0.64
0.95
0.79
d- d-Othello
0.96
0.99
0.96
d-Pusher
0.96
0.99
0,96
d-Push Your Luck
0.23
0.48
0.16
d- Red Finned Blue Eye
0.72
0.84
0.70
d-Sailing Wind
0.70
0.92
d S-vvinng
0.96
0.96
0.96
d-Skills Teaching
0.59
0.65
0.59
d-SysAdmin
0.15
0.48
0.20
d Tamaaiisk
0.35
0.56
0.39
d- d-Traffic
0.53
0.54
0.53
Wildfire
0.19
0.37
0.30
d- diWildiee Preserve
0.06
0.08
0.08
Parameter-optimizzed KVDA-UCT; Next, we compared KYDA-UCT, (0,0)-OGA (i.e. standard OGA-UCT), and (E,, 00- with E, > 0 in terms of their parameter-optimized performances deterministic environments. For all methods, we optimized over C' e {0.5, 1,2, 4, 8, 16) and some domain-specific values for e, that listed in the supplementary materials in Tab. Each parameter
8

---

Published as a conference paper at ICLR 2026
Table 2: Average returns (1) using 100 (right) and 1000 (left) MCTS iterations for KVDA-UCT, (oo, 0)-GA, (0, 0)-OAA (i.e. OGA-UCT), and the maximal performance of (ea, 0)-OGA when varying & > O. In the supplementary materials in T the comain--ppecife ea-values are listed. All performances are the maximal performances obtained by varying the exploration constants Ce (0.5, 1,2, 4, 8, 16]. Note that KYDA-UCT (our method) outperforms OGA-UCT in most en- vironments. Furthermore, KVDA-UCT is either equal performs better than parameter-optimized (Ea, 0)-OGA even though KVDA introduces no extra parameter.
Trerations
(f,-0)-
KYDA (Omr)
KYDA
Domain
OGA
0GA
OGI
OGA
0GA
(ouns)
B.Consecta
12.7=
47.5
24.8
A.Cinerritoo
96.1
96.0
84.1+004
Othello
181.2
100.2 0.8
180.7 1.5
170.6
154.7 1.3
138.8 1.08
155.1
15L.
Pusher
104.3 0.0
104.5
104.5
101.5.1..
104.3 0.0
104.3 1 0.0
104.5
104.3
0.0
d-Actdemic Advising
99.9 0.2
40.0
-000 +0.0
-40.1 -0.1=
-666 0.7.7
-66.2 0.6
-56.2
-50.1 -50.1.+000 0.6
d-Curpentive Recon
16.1=0.1 0.1
0.1
1..2
16.0 0.1
10.++003
10.0++0.3
11.0=0.
11.0
d-Farth Obsrration
-7.18
0.11
-30.0 0.4
-500
-7.02 0.110
-1022 0.2
-28.4++0.2
-98.4 0.2
-7.45 0.12
Elevatios.
0.4
14.22 0.3
1140 1.3
- -1.99 1...1..4. 0.4
-18.0
-18.1 0.3
-17.9
0.3
-18.0 0.3
d-Gamm of Life
606.7
2.5
0646 2.5
665.8 +:2.5
664.8
2.6
642.1 2.2
642.1
2.2
641.9 2.1
Manufictuurrr
1200.6 35.0
1638.4 22.21
12060
1168.2
19.4
16.9 14255.1
168013 2 22.4
18826 6: 2012
23.2
d.Push Your Lack
125.1
06.7
+-2.4+2..5
157.9 - 2.2
105.5 1.5
00.3++00
107.0 -
107.8 1.5
$1915 44.7
7500.0
795066
8220.8
46.55
3338
1664.13344
7486.1
7098.2
Sailing Wind
-..
-30.1 1.0.6
58.9
37.7 0.6
-64.7 1.0.8
169
-640
63.6 0.8
66.0
63.4
06
0.2
d-Stills
207.9
210.2:::4.5
1500++30
158.5+39
A-Syxndiin
477.1+133
-28.22
2.7
-60.0
Traftiic
1.54
0:07
0.07
9.21
9.21
9.21
0.22
-9:21
0.22
Wildllie
500.5
-15.0
86.1
179.9
191.1
008.1 8611
-179.2 96.1
A-Willifee Preserve
1388.0++08
1387.0++0.7 0.7
1357.9 0.7
1388-5
2:
1388.9 2:
1388.9 0.9
1388.9
1388.9
combination was evaluated on all here-considered deterministic environments with a budget of 100, 200, 500, and 1000 MCTS iterations. Tab. list the performance for 100 and 1000 iterations, the ta- bles for 200 and 500 iterations are found in the supplementary materials in Tab. [ and G The results clearly show that KVDA either outperforms all other competitor methods or tightly within the confidence bounds. There are a number of environments such as Manufacturer, Tamarisk, or Push Your Luck where KVDA-UCT simultaneously outperforms all competitor methods at once. Figure 2 plots the performances in dependence of the iteration budget for these tasks. The performance graphs for all other environments are found in the supplementary materials in Section A.7
-1160
140
135
Average Return
-1200
Averagee Retum
-220
Average Return
130
1250
-240
125
1300
120
-200
OGA-UCT KVDA-UCT De,, 01-OGA
115
OCA-UCT KVDA-UCT
-1400
RVSAJUCT
-300
110
Ie., PHOGA
105
OG-GGA
-1450
1000
100
200
5.00
1020
200
500
1000
Count
Iteration Count
Iteration Count
(a) d- d-Manufacturer
(b) d-Tamarisk
(c) d-Push Your Luck
Figure 2: The performance plots in dependence of the iteration budget of parameter-optimizee KVDA-UCT (our method), OGA-UCT (Anand et al. 20166), and (e,, 0OAA on selected environ- ments. KVDA-UCT clearly outperforms its competitors. The plots for the remaining environments can be found in the supplementary materials Section A.7
Single-paramette KYDA-UCT: Next, the generalization capabilities of KVDA-UCT in com- parison to OGA-UCT and (ea, 0)-A on deterministic environments were tested. Like last section, we ran all algorithms using 100, 200, 500, and 1000 MCTS iterations whilst varying C' E (0.5, 2, 4, 8, 16} and ε, with domain-specific values (see supplementary materials Tab. 4 Instead of considering the best-performances. we calculated the normalized pairings score for each parameter combination (except for (E., 0)-OGA where we considered the maximum performance across all E, 0 values single parameter combination), constructed over the tasks which the pairs of all environments and iteration budgets. Bar 3 the 6 agents with the highest score well as the agent with the lowest score. Both the top spots are occupied by K KVDA-mette -Dm-moh

---

Published as a conference paper at ICLR 2026
followed by OGA-UCT and (Ea, OGA. The best performing algorithm overall is KVDA-UCT with 4.
0.6
0.531
0,527
0.4
0.324
D 322
0.276
0.254
0.2
Pairings score
0.0
-0.2
-0.4
-0.6
-0,670
-0.8
C=2, KVDA-UCT C=2, OGA-UCT
C-4, (E,.0)-OGA C-0.5, (E,,0)OGA
C=4, KVDA-UCT
C=4, OGA-UCT C-8, KVDA-UCT
-1.0
Algorithm
Figure 3: The normalized pairings score (f) for the top and the worst agent on deterministic environments. The agents considered were KVDA-UCT (our method) which performs best overall, OGA-UCT (Anand et al., 2016). and (ea, 0)-OGA (Schmöcker et al., 2025d). En >> with the exploration constants C' {0.5, 1, 2, 4, 8, 16) and budgets of [100, 200, 500, 1000] iterations. The top two spots are occupied by our method KVDA-UCT, with the best overall performing algorithm being KYDA-UCT with C = 2.
E--VDAA on stochastic environments: Lastly, the performance of & KVA on stochastic envi- ronments were tested by rumning both E,-KDDA and (Ea,
the stochastic versions of the here-considered environments. We considered the iteration budgets of 100 and 1000 iterations and varied C {0.5, 2, 8, 16), {0, 0.4, 0.8. 1.2, 1.6) and the same domain-specific e, that were used the previous sections whose values are listed in the supplementary materials Tab. 4 For each environment, Tab. of the supplementary materials lists the parameter-optimized performances of both algorithms. In contrast to the deterministic setting, KVDA does not outperform OGA. For most enviromments both perform equally well with some exceptions such as Manufacturer were Er" KVDA clearly performs best and Tamarisk were (Ea, -GG performs best. We believe mediocre performance of e, stems from the fact that since e,KVDDA essentially ignores immediate rewards when it builds abstractions, the number of faulty abstractions that occur in this approximate setting is increased. In the deterministic setting, no faulty abstractions are built.
CONCLUSION, LIMITATIONS AND FUTURE WORK
This paper introduced KVDA-UCT, an extension to OGA-UCT, that additionally groups states and state-action pairs that do not have the same Q" or V* value as long as the difference is known. We evaluated and compared KYDA-UCT where KVDA-UCT outperforms all its competitors both in terms of generalization well as the parameter-optimized performance. This does hold true for the stochastic setting for which generalized KVDA-UCT to E,-A which is only clearly better than (e,, &)-OGA in few envi- ronments.
OGA-UCT and (e., 0)-OGA
deterministic setting
A first avenue for future work will be to further investigate the reasons for the mediocre performance of the experiments in this paper of the stochastic setting and develop an extension E suited for this setting, Another limitation of KVDA abstractions in its current form is its limitation to MDPs (i.e. single-player games). Even though MCTS is principle applicable to multi-player games, KVDA is not because in any state, there is in general, no unique V value from which differences can be built, as there are potentially many equilibria with different payoff vectors. Future work will be extend KVDA to this setting.
10

---

Published as a conference paper at ICLR 2026
7 REPRODUCIBILITY STATEMENT
In our experiment setup, we have a subsection called Reproducibility in which we provide down- load link the full codebase used for this project well as compilation details. The codebase contains an elaborate README detailing the steps to reproduce the experiments.
REFERENCES
Ankit Anand, Aditya Grover, Mausam, and Parag Singla. ASAP-UCT: Abstraction of State-Action Pairs UCT. In Qiang Yang and Michael J. Wooldridge (eds.), Proceedings of the Twenty- Fourth International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Ar- gentina, July 25-31, 2015, pp. 1509-1515. AAAI Press, 2015. URL http: jcai. org/
Bbstract/15722
Ankit Anand, Ritesh Noothigattu, Mausam, and Parag Singla. OGA-UCT: on-the-go abstractions in UCT. In Proceedings of the Twenty-Sixte International Conference on International Confer- ence on Automated Planning and Scheduling. ICAPS' 16, pp. 29-37. AAAI Press, 2016. ISBN 1577357574.
Cameron Browne, Edward Jack Powley, Daniel Whitehouse, Simon M. Lucas, Peter Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez. Liebana, Spyridon Samothrakis, and Simon Colton. Survey of Monte Carlo Tree Search Methods. IEEE Trans. Comput. Intell. Al Games, (1):1-43, 2012. doi: 10. 201/T ICIAIG.2012. 2186810
URL
11099
Marek Grzes, Jesse Hoey, and Scott Sanner. International Probabilistic Planning Competition (IPPC) 2014. In Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), 2014.
Jesse Hostetler, Alan Fern, and Thomas G. Dietterich. Progressive Abstraction Refinement for Sparse Sampling. In Marina Meila and Tom Heskes (eds.), Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, UAI 2015, July 12-16, 2015, Amsterdam, The Netherlands, pp. 365 374. AUAI Press, 2015. URL http://aual http://aaaa..orr//u12011555 ortguu1201///
pxoceediinsspppperee /8t.pd1...p... pdf
Nan Jiang, Satinder Singh, and Richard L. Lewis. Improving UCT planning via approximate ho- momorphisms. In Ana L. C. Bazzan, Michael N. Huhns, Alessio Lomuscio, and Paul Scerri (eds.), International conference on Autonomous Agents and Mulli-Ageni Systems, AAMAS '14 Paris, France, May 5-9, 2014, pp. 1296. IFAAMAS/ACM, 2014. URL http://dl. c
acm. www.orr//ooo
Levente Kocsis and Csaba Szepesvári. Bandit Based Monte-Carlo Planning. In Johannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou (eds.), Machine Learning: ECML 2006, 17th Euro- pean Conference on Machine Learning, Berlin, Germany, September 18-22, 2006, Proceed- ings, volume 4212 of Lecture Notes in Computer Science, pp. 282-293. Springer, 2006. doi: 10.1007/1187184211 29. httpp:tt http:///ooooorg//./../
B. Ravindran and A. G. Barto. Approximate Homomorphisms: A Framework for Non-Exact Min- imization in Markov Decision Processes. In Proc. Int. Conf. mhowl.-Based Comput. Syst., pp- 1-10, 2004.
S. Saisubramanian, S. Zilberstein, and P. Shenoy. Optimizing Electric Vehicle Charging Through Determinization. In ICAPS Workshop on Scheduling and Planning Applications, 2017.
Robin Schmöcker. Grouping nodes with known value differences: A lossless UCT-based abstraction algorithm. 2025, Repository available at: battp:://gitut com./codebro634//0..
UCT git
Robin Schmöcker and Alexander Dockhorn. A survey of non-leamnigg based abstractions for quential decision making. JEEE Access, 13: 13:100888-10883 2025. doi: 10.1109/ACESSS2222. 3572830.
11

---

Published as a conference paper at ICLR 2026
Robin Schmöcker, Alexander Dockhorn, and Bodo Rosenhahn. Aupo abstracted until proven otherwise: A reward distribution based abstraction algorithm, 2025a. URL https::/arxat
/tg/abs/2510. 23214
Robin Schmöcker, Alexander Dockhorn, and Bodo Rosenhahn. Accelerating probabilistic planning research: High-speed implementations of stochastic markov decision processes and their catalo- gization. IEEE Access, 33199333--19466 2025b. doi: 10. A0.10//AACESSS22255.611555
Robin Schmöcker, Alexander Dockhorn, and Bodo Rosenhahn. Investigating intra-abstraattion poli- cies for non-exact abstraction algorithms, 2025c. URL httpss//aaxx....orr//a/////////
24297
Robin Schmöcker, Lennart Kampmann, and Alexander Dockhorn. Time-critical and confidence- based abstraction dropping methods. In 2025 IEEE Conference Games (CoG), 2025d. doi: 10.1 10.11099C006647522225.. 11122222..111121111 1114261.
Robin Schmöcker, Christoph Schnell, and Alexander Dockhorn. Investgatiinngnnnntioooooo scale independent uct exploration factor strategies, 2025e. BBLL:ttpps///rxx..oorr//b/s/ 2
David Silver, Thomas Hubert, Julian Schrittwieser, loannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy P. Lillicrap, Karen Simonyan, and Demis Hassabis. Mastering Chess and Shogi by Self-Play with a General Rein- forcement Learning Algorithm. CoRR, abs/1712.0181s. 2017. URL http::/p://// //apivv//rxvv.../ orgg
0tt5/172.01111
Samuel Sokota, Caleb Ho. Zaheen Farraz Ahmad, and J. Zico Kolter. Monte Carlo Tree Search With Iteratively Refining State Abstractions. In Mare' Aurelio Ranzato, Alina Beygelz- imer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Informa- tion Processing Systems 2021, NewrIPS 2021, December 6-14, 2021, virtual, pp. 18698 18709, 2021. URL https://proceednngg
pctp:/paape//22//aass... 9bCead00a2 1ee.eeaaalllaa 2eaaacc122e0a1222e4422333 Abstract html
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. The MIT Press, 2nd edition, 2018
Linjie Xu, Alexander Dockhorn, and Diego Perez-Liebana. Elastic Monte Carlo Tree Search. IEEE Transactions on Games, 15(4)5527-537, 2023. doi: 10. 2003332.00233222555..
Sung Wook Yoon, Alan Fern, and Robert Givan. FF-Replan: A Baseline for Probabilistic Plan- ning. In Mark S. Boddy, Maria Fox, and Sylvie Thiébaux (eds.), Proceedings of the Seven- teenth International Conference on Automated Planning and Scheduling, ICAPS 2007, Prov- idence, Rhode Island, USA, September 22-26, 2007, pp. 352. AAAI, 2007. URL http: php
7/ Ltt.aaaas///LLaaaa//////////////
Sung Wook Yoon, Alan Fern, Robert Givan, and Subbarao Kambhampati. Probabilistic Planning via Determinization in Hindsight. In Dieter Fox and Carla Gomes (eds.), Proceedings of the Twenty- Third AAAI Conference on Artificial Intelligence, AAAI 2008, Chicago, Illinois, USA, July 13-17, 2008, pp. 1010- 1016. AAAI Press, 2008. URL http: www aaai, www./orggg/ibb.... AAAI2200//aa10-16666
12

---

Published as a conference paper at ICLR 2026
A SUPPLEMENTARY MATERIALS
A.1 KVDA PSEUDOCODE
Algorithm 1: S--VAA
Parumeters: agg.args
Input:
Globals: maxzid, abstractionsQ abstractionsiaate,, RecencyCount, de d.(())
imit.. ttrrressuutt
for to oga argsiiteniions do
va lues abstractiionn( + 4, obstractionss(Q represetativv)) are used
leaf. Jeaff, newQuodes ttreFolicyymee
foreach newQnode new-modes do
abstractionsQ [hstratiims@[uoo.uuudd]]]] abstroctions Dnseractioonnnneannnaae (neeQnode)] representative newQnode newQnode mix_id+++
nww.si.ngleton...bbottre...
teaf then abstractiosSttee(llaff new www..ainglassee www.innngwwonnnnnooooooooooooooe att.....tte.llasrratiiooo.
10
11
abstractionsSsate (feass representative = leaf
12
ba backup (path so Jeaf sollout foreach (Q.s) E path leaf do
13
14
abstractions((() valuue
d,(0, www.saaciioollllompeoeeooe
15
16
foreach (Q.s) deaf do
17
update Q abstraction: (Q)
18
return arg max Q(state;
19
function www.eeeeeeee....sseerrtoioooooo.
20
if ReceecyCoouu((
K then
21
return
22
RecencyCount [Q] new abs Stapputte..bbsttrttionnn..
23
24
old dist 4,(0 abstractionns (Q) representative))
25
update Q, new abs, abt. representative) using Equation
26
if abstractions @ # abs d,, (new abs, representutivee, @ # dlss then
27
abstractions Q visits
Qivisis; new.abs, visits Qisists
abstractions Q value Q-volue +.visssss d, abstractionns represennativve)
29
new.abs. value + Q value +is. d.l., new repeessaepe))
absruccionss(0) aeaerrrttiomlqQ....eeeeenaatieee seprettaattee and abstractionsQ(U). > Q...r choose, att wew ylabsractionu]])) excludingg)))
31
31
recalculate affected da values
.33
abstractions Q value obstractions[(Q) visits da Q...)
34
abstractions new
(x,
36
13

---

Published as a conference paper at ICLR 2026
Alg Algorithm 1: e (continued)
48
function updote_state abstrrattttt (slat)
49
newaabs = compute_stat
// ebstracts two states if:
ans #2)
old dist (state, abstractionSSaatsssuu hatpa/sssssaaeeepppeaaaaaa
51
d, (state, new Jhs represenaativv)) get stttcdittanneeatt nww.pws..sseeeeoeeoo.o.ee...ssss www.phhs.epresenttttee))
52
old.dist if new.obs * abstractionstaaee (state) or d, (state, new abs.abbb..see.. res.reerreentatvv)) * old dist then
53
if sttate abstractionsStutejj [state] representativv and abstractiomsSttte
then
54
choosa-nat new represent randomlyy y (absrrattonnnSStteet statej excluding state)
recalculate affected d, values
56
abstractionsSitte (sttesss new abs wtt.sat._.abstrttiioooio..o parents)
57
function get.sistance 181, #2)
59
if #1 abbtractActionn o abstractActionns then return
foreach abe absstrecttction do
62
if { da(O, repreenttn)) E abrQ s1.actions]
}| >
dalQ, vap@ssvprss)
Q abeQ O s2.actions
} > then
return
64
distances
65
foreach abrQ atsettmettctioonn do
66
distances. (d. (p11.22))
11 E abrl, pı E Esaactions, E sg.actions
67
if distances.siz > then return oc else return d distances m (d)
14

---

Published as a conference paper at ICLR 2026
A.2 PROOF THAT d du AND d: de
In this section, it will be shown that the KVDA difference functions d, and d, at any step (including the step of convergence) coincide with d; and d for state or state-action pairs within the same abstract state. This will be proven inductively. First, note that this statement is true for the base case as all terminal states have value of V = O; hence, their differences are also For the following induction steps, assume that d, is bootstrapped by d' and d, is bootstrapped by da. Next, let (p1,P2) (22) & H. By definition, holds that
d' (p1,p2) = R(p2) R(pi) + > (P(s) p2) - P(s)pi))V
sES
By rewriting (s) in terms of the difference to its abstract representative, obtains
L= Σ Σ
- ( - (5's', =o² induction
ZEt 8'E
= -(s
ZEX ExE
+ Σ>
(P((') pi) PP p2)) ds's', sz)
#EX d'Em
(8)
= Σ V*(5z)
Σ (P(5') P(s p1))
ZEX
=0
(P(6') PP('' (8(()')))
EEN E'Ex
(p(11 p2) + R(p1) - R(p2).
Hence (2)) ds (2), Note that this proof holds for any choice of the abstract nodes' representatives. Lastly, let (81, 82) E & and 82()188) = d. First note that E arg max as), and ((s), (s,,
H' with d,((s,,0) a), (s2.a) d for some
d'EA(a1)
Q"(s2, a') because for all A(s2) holds that à e A(s2) then à arg a'EA(e2)
(* a) Q' (s), ()((, (s1,
2 Q' b) d= (8),a)))
(9)
[22 b) d[((s,, (), (81, b))
d=2( (s1,
= Q" (82, b) (-d) + d = Q" (22.
where ((s1, 6). (82, b)) H' with di ((51, 8), (52, b)) d. And since V*(s) max Q*
(s
aEA(s)
directly follows that d d 82) dx 82),
n.
A.3 RUNTIME MEASUREMENTS
Tab. lists the average decision-makinn making times for each environment of KYDA-UCT compared to OGA-UCT for 100 and 2000 iterations states sampled from distribution induced by random walks. The numbers show a median runtime overhead < 1% for 100 iterations and 1% for 2000 iterations.
15

---

Published as a conference paper at ICLR 2026
Table 3:
Average decision-makee making times in milliseconds of KVDA-UCT compared OGA-UCT
This data was obtained using an Intel(R) Core(TM) i5-9600K CPU @ 3.70GHZ.
Domain
KVDA-100
OGA-100
KVDA-2000
OGA-2000
Academic Advising
3.85
3.82
133.82
131.74
Cooperative Recon
5.26
5,23
221.98
220.19
Earth Observation
7.54
7.54
197.38
199.13
Elevators
6,85
6.79
231.93
227.69
Game of Life
4.99
4.96
139.38
134.06
Manufacturer
10.26
11.01
281.03
294.91
Red Finned Blue Eye
6.69
6.66
152.46
151.56
Sailing Wind
4.11
4.09
139.39
140,82
Saving
3,75
3,71
124.22
111.13
Skills Teaching
4.97
4.94
219.90
216.68
SysAdmin
6,27
6.20
150.19
152.72
Tamarisk
4.65
4.64
167.23
148.03
Traffic
5.44
5.42
161.33
159.35
Push Your Luck
5.84
5.78
170.25
175.19
Wildfire
7.46
7.43
368.66
476.95
Wildlife Preserve
7.06
7.05
2020.35
1685.53
Othello
18.63
18.67
434.88
428.34
Connect4
4.40
4.33
133.35
123.58
Constrictor
23.23
23.10
616.38
611.72
A.4 DOMAIN-SPECITIC VALUES
One problem with (E)-AAA is that while is neatly bounded by 0 and 2, the E, value has to be chosen on a per-environment basis since for example the value c 0.5 would be equivalent to 0 for any environment with rewards that are all greater than 0.5. Tab. lists the hand picked values for each environment that we chose for the experiment section. The values were chosen to be at the boundary of rewards that actually occur in these environments.
Table 4: A list of the environment--peeiffc E values that were evaluated in the experiment section for (e,, 0)-AGA All domains that not explicitly listed here, used the default values e, {1, oo).
Environment
e, values
Academic Advising
0O
Wildlife Preserve
5, 10, 20, oc
Red Finned Blue Eye
100, 200,
Wildfire
5, 10, 100,
Push Your Luck
2, 5, 10,
Skill Teaching
2, 2, oc
Tamarisk
0.5, 1.0, oo
Cooperative Recon
0.5, 1.0, oo
Manufacturer
10, oc
Connect
1, 10,
Othello
5, 10, 20,
Constricior
10, 20, 30, oo
Default
1. 0c
16

---

Published as a conference paper at ICLR 2026
A.5 PERFORMANCES OF EV-K
Table 5: The parameter-optimizer performances of <--VDD (our method) and (E,, 4I-OGA on the stochastic versions of the sere-presented environments using 1000 MCTS iterations (left) and 100 MCTS iterations (right). The environments Push Your Luck and Wildlife Preserve from the deterministic experiments were excluded computing their transition probabilities which is quirement for both 5-KVDA and (Ea, 5)-OGA is infeasible. Furthermore, Red Finned Blue Eye, Wildfire and Elevators were also excluded as the performances" variances were high in these set- tings such that obtaining low confidence bounds was infeasible. Contrary to the stochastic setting, V-KVDDA does not consistently perform better than OGA. For example, while Ev-VDA can gain clear advantage in Manufacturer and Sailing Wind, (E,, GGGA is significantly better in Tamarisk.
Average returns (1) for 1000 MCTS itera- tions
Average returns (1) for 100 MCTS iterations
Domnin
6 Rv-AVDA (ours)
Domain
(5,,4))00AA
5,KKDDA (ours)
s-Academic Advising
-88.0
-8.88 1.2
Academic Advising
0.3
63.9 + 0.8
Cooperative Recon
7.08 - 0.37
7.07 0.35
Caoperative Recun
0.2
s-Earth Observation
-13.7
-138
>-Earth Observation
-7,97
-7.95
0.23
s-Game Life
498.7
497.6=3.0
572.9
s-Manufacturee
-1457.6
-1159.6
-Manufacturer
11.
863.3
10.7
s-Sailing Wind
-18.3+1.1
-73.2
s-Sailing Wind
1.3
1.3
s-Saving
4..6
44.5 +4.5+00.2
S-Saving
50.7
50.6
s-Skills Teaching
-.94
-9.00
S-Skills Teaching
71.1
6.9
S-SyAdmin
332.7
331.1
SysAdmin
2.0
403. 2.1
Tamariss
-767.9
-36.1
Tamarisk
8.2
8.3
Trafffic
-24.
-22.0 + 0.4
T-raaffic
-.3
-3366
A.6 PERFORMANCES OF PARAMETER-OPTIMZZE KVDA-UCT FOR 200 AND 500 ITERATIONS
Table 6: Average returns (t) using 500 MCTS iterations for KVDA-UCT, (co, 0)-AA, (0, 0),
OGA (i.e. OGA-UCT), and the maximal performance of (ea, 0GA when varying > 0. In the supplementary materials in The -omaii--peeific valuue are listed. All performances are the maximal performances obtained by varying the exploration constants C E (0.5, 1, 8,8, 16). Note that KYDA-UCT (our method) outperforms OGA-UCT most environments. Furthermore, KVDA-UCT is either equal performs better than parameter-optimizee (Ea, 0)-OGA even though KYDA-UCT introduces no extra parameter.
Domain
(0,0)-OGA
(xo, 0)-GA)
(e,, 0)-OGA
KYDA-UCT (ours)
Connect4
36.7 0.5
41.3 1.0
41.3 1.0
39.9
Constrictor
94.2 0.2
92.8.1.0
94.2.0.33 0.3
94,0
Othello
175.3 + 1.1
1.6.8
175.21116
173.411.1
Pusher
104.5
0.0
10..500000
104.5
d-Academic Advising
0.3
0.3
0.3
4.3
d-Cooperative Recon
1.13
1..4...1
1.24
15.3
d- 1-Earth Übservation
0.12
-.4
-.4
-698 0.10
d- E-Elvvators
-.3
-
0.3
14.9
Game of Life
659.9 6.0
650.9
660.7122.
661.612.
Manufaacturer
-1270.2
-1646.1120x
1259.11
-1178.3 22.3
d-Push Your
124.2
6.0
1.1.4220
136.5 +
d-Redd RedFFinneBBlueEye
7978.4
7738.11134.2
34.
8041.7 1: 33.1
d- d-Sailing Wind
0.7
-.7
-42.8
-41.6 0.7
d-Saving
64.5 0.2
6.27+
6.22+0.2
6..4
d-Skills Teaching
202.8 3.7
201.0
202.6
206.5 1 3.7
d-SysAdmin
4.7.
4.8
4.0.8
478.1
d-Tamarisk
-228.9
-217.9
-97.22 0.09
-2.7.9+2.4
d-Traffic
2..0 0.10
"2.5310.09
2.53 0.09
d- Wildfire
-183.6 36.1
-506.0 36.2
-409.6
-19066
d- Wildlle Preserve
1385.1 + 0.8
1384.2 + 0.8
1385.5 - 0.8
1385.0 + 0.8
17

---

Published as a conference paper at ICLR 2026
Table 7: Average returns using 200 (1) MCTS iterations for KVDA-UCT, (co, 0)- (0, 0)- OGA (i.e. OGA-UCT), and the maximal performance of (E-,0) when varying the supplementary materials in the domain-speen speaiii---pificccc va-values are listed. All performances are the maximal performances obtained by varying the exploration constants C' E (0.5, 1, 4, 8,8, 16). Note that KVDA-UCT (our method) outperforms OGA-UCT most environments. Furthermore, KVDA-UCT is either equal performs better than parameter-optimizeed (ea, 0)-OGA even though KVDA-UCT introduces no extra parameter.
0. In
Domain
(0,0)-OGA
(oo, O)-GAA
()-OO)
KVDA-UCT (ours)
Connect4
29.1 0.4
29.9 + 0.6
29.9 0.6
29.5 0.4
Constrictor
91.3 + 0.3
0.2
91.3 0.4
91.2 0.3
Othello
164.5 1.2
152.1
164.7
164.5+12 1.2
Pusher
104.5
0.0
0.0
104.5
d-Academic Advising
0.7
4.7 0.4
-47.5
d-Cooperative Recon
133
1.3
0.3
1.3
d- 1-Earth Observation
0.14
0.2
0.2
-7.11 0.11
d- Elevators
0.3
-.3
-.3
-16.0
d- 4-Game of Life
2.1.44
652.3 2.4
6.2.3
2.5.1
d- A-MMaufactuuer
15.70.7
-1641.4
1320.774
-1180.8
22.2
d-Push Your Luck
1.2
6.91
129.6
125.9
7713.6
34.7
7513.4 34.9
7716.9 33.7
d-Sailing Wind
0.8
-.2
-.2.3
-51.7
d-Saving
6.2
5.2
0.2
0.2
d-Skills Teaching
1.9
1.8.
186.7
185.0
d-SysAdmin
4.6.
4.0.2
450.3 1.2
478.9 1.4
d-Tamarisk
5.5
3.5
3.5
3.2
d- Traffic
0.15
0.15
0.15
-..34
d- d-Widdfire
-193.8 36.1
-476.5
-432.2
-
d-lidlle Preserve
1390. + 0.9
1389.4 0.9
1389.4 + 0.9
1389.5 + 1.0
18

---

Published as a conference paper at ICLR 2026
A.7
PERFORMANCE GRAPHS OF PARAMETER-OPTIMEZZ KVDA-UCT
Aesrage Retomn
Aesrrgg Retum
8
%
100
benation
1000
190
Penation Count
130
190
becstion
196
Renution Courtt
(a Elevators
(b) Earth Observation
(c) Game of Life
(d) Manufacturer
Berition Count
200
500
1003
Beretion Count
Eeratin Count
gg Red Finned Blue
(e) Push Your Luck
(f) Cooperative Recon
(h) SysAdmin
Eye
Asergg Retom
Avsrgge Roar
-23
Retum
Beretion Count
Remation Count
1200
Baratton Count
230
500
100
(i) Skill Teaching
(i) Sailing Wind
(k) Tamarisk
(I) Traffic
15
Austge Rature
Rature
1307
Heration
fteratonn Count
Beration Count
180
Heration
50G
(m) Wildfire
(n) Wildlife Preserve
(0) Connect 4
(p) Constrictor
Avange Retumn
OGA-UCT KVDA-UCT (Ea, 0)-OGA
1048
1XC
206
50
100
Recition Count
Berason Count
Legend
(q) Othello
(r) Pusher
(s) Saving
Figure 4: The performance plots in dependence of the iteration budget of parameter-optimmzzed KVDA-UCT (our method), OGA-UCT, and (ea, 0)-OGA, E, > 0 all considered environments. Our method KYDA-UCTI is either always tightly within the confidence bounds of the top-competitor method outperforms all competitors simultaneously.
A.8 MONTE CARLO TREE SEARCH
This section specifies the MCTS that was used for the experiments of this paper.
1. Our MCTS version builds a directed acyclic graph instead of a tree, ie. different state- action pairs in the same layer lead to the same node. This is necessary condition for ASAP and KVDA to detect any abstractions since they bootstrap the search graph converging.
19

---

Published as a conference paper at ICLR 2026
2. We use the Upper Confidence Bounds (UCB) tree policy which is defined as
log
Na' a (A())s)
Σ
V N
UCB(a)= =
+
(10)
N
Q term
Exploration
where, is the state at which the decision has to be made, Va is the sum of returns of the action under consideration, and N₂ are its visits. The tree policy chooses the action that maximizes the UCB value.
We use the greedy decision policy, which choosing the root action, after the search statis- ties have been gathered, with the maximum Q value.
A.9 PROBLEM DESCRIPTIONS
The descriptions for all the problem models considered here are described in detail by Schmöcker al. [Schmöcker al. (2025b). Most models are parametrized (e.g. choosing concrete racetrack for Racetrack), the concrete instance used for the experiments is found in the Experiment
folder in our publicly available implementation (Schmöcker 2025 Next, we describe the heuristic functions used for the two-player games.
1. Connect 4: As the heuristic from the perspective of player one is given by
+ 5mg + 25n
(11)
where n, the total number of stones that are in one row/columm/diagonall divided by i, but which not part of row/column/diagonal of size i + 1.
2. Constrictor: The heuristics function used for player one is the number of grid cells that player one could reach before its opponent. If player one wins, the value 100 is added to the heuristic.
3. Othello:
The heuristic function for any state for player one is given by weighted sum of all the occupied grid cells. The weight of cell (x, y) is given by 10/(1 + d) where is the distance to the closest corner cell. The weight is positive for any cells occupied by stones of player one and negative for those occupied by player two. In the round player one wins, 100 is added to the heuristics value and 100 if player two won.
4. Pusher:
We used the heuristic for player one that is equal to the difference of alive units of player one and player two in addition to the Manhatten distance between player one and player two's 's units' center of mass. In the turn that player one wins, 100 is added to the heuristic value.
20


Reviews:
[{"review_id": "vFnyDdIyOv", "reviewer": "Reviewer_VwR2", "summary": "The paper tackles the problem of low sample efficiency in Monte Carlo Tree Search (MCTS). Existing methods, such as OGA-UCT, improve efficiency by abstracting similar states and state-action pairs, but rely on strict equality of immediate rewards, which limits the scope of abstraction. The authors propose a new framework, Known Value Difference Abstractions (KVDA), which allows grouping states or actions with different but inferable value differences. Incorporating KVDA into OGA-UCT (forming KVDA-UCT) leads to more abstractions, better efficiency, and improved performance across various deterministic environments, all without introducing extra parameters.", "questions": "- In the deterministic setting, when might the proposed abstraction (based on the known difference in values) harm the performance?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The problem tackled by this work is well motivated. Increasing the number of abstractions helps improve the sample efficiency of MCTS.\n- The solution, proposed in this work, sounds. The idea of abstraction using a known difference in values between state(-action) pairs is novel to me.\n- I really appreciate Section 2, which is essential to understand the preliminaries and know the related work.\n- I believe the results are strong and the proposed abstraction technique does not hurt in terms of performance, except in the stochastic case.\n- The limitations are clearly mentioned.", "weaknesses": "- I believe some parts of the main paper need to be adjusted for clarity.\n- There are some tables and figures that are placed in the Appendix while being discussed as main results in the main paper. I understand the issue with space, but I believe it is really important for the main paper to be isolated from the appendix. \n- This is especially because the appendix is not well-presented, since there are some figures that are not inserted correctly.\n- I believe the baselines benchmarked in this work are fair, but I would be interested in looking into some other methods that were already discussed and mentioned in Section 2.\n\n**Minor Issues:**\n- There is an issue with referencing the sections in lines 376-377.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UWRCLu2NcS", "reviewer": "Reviewer_iyK8", "summary": "Authors propose KVDA, a novel MCTS lossless abstraction method rooted from ASAP. KVDA modifies ASAP’s abstraction criteria, especially removing the criterion of immediate reward equivalence, and leaving the Q differences between the Q nodes as a record when abstracted. For state nodes, KVDA abstracts two state nodes if their all Q nodes are abstracted to each other and Q differences of their all abstractions are same. Authors merge KVDA with UCT as KVDA-UCT, then compare abstraction ratio and general performance of KVDA-UCT with OGA-UCTs on various board games, proving that KVDA-UCT leads more abstractions than OGA-UCT, and that KVDA-UCT outperforms OGA-UCT mainly on deterministic environments.", "questions": "All questions and suggestions are included in the Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 1, "strengths": "The idea is distinct and sound, supported with theoretical guarantees. Background description is elaborate and well grounded. Authors adopt adequate counterparts for comparison experiments, which are conducted on appropriate domain and various tasks. They also consider extending KVDA to stochastic environments for generality.", "weaknesses": "#### Major Weaknesses\n\n- KVDA–OGA Performance Ambiguity: The biggest concern is that according to Table 2, different from the authors’ comment, the performance of (ε_a, 0)-OGA with 1,000 iterations seems to generally outperform the KVDA. If it’s the case, authors might claim that since KVDA more generously abstracts than (ε_a, 0)-OGA, KVDA aggressively finds good paths more efficiently under the condition of low iteration budget. Here, at least I can agree that KVDA is better than (∞, 0)-OGA, since (∞, 0)-OGA would yield lots of faulty abstractions while KVDA would not because it’s lossless. This is where KVDA mitigates the potential negative effects of its generosity and turns it to its advantage. Still, if (ε_a, 0)-OGA performs poorly in lower iterations but achieves the best performance at 1,000 iterations, it may imply that the conservative aspect of OGAs contributes to steady but ultimately powerful performance growth. Since 1,000 iterations are not practically high budget, if this trend reproduced in higher iterations, the contribution of KVDA would become more limited, unless OGAs only outperform in practically very high budget. Because KVDA struggled in stochastic envs (and I agree leaving improvement of it for future work), proving the effects of KVDA in deterministic envs would naturally be emphasized to highlight the sufficient significance of the paper.\n- Insufficient Method Discussion: Secondly, the lack of discussion on the main method makes authors’ suggestions less convincing. Throughout the paper, it appears relatively limited compared to other parts such as background descriptions and equations. Although introducing the foundations and ASAP (including subsection 3.1) is necessary (But are Sections 2.3 and 2.4 really necessary as standalone subsections?), it would be more convincing and provide denser insight to focus more on the main proposal, especially by providing theoretical or empirical analysis (e.g., a few actual abstraction examples obtained from the experiments) of the expected positive/negative effects due to the KVDA’s lossless but generous abstraction conditions. Explanation of why the results were so good for some tasks and vice versa might also help.\n- Poor Structural Cohesion: Lastly, the overall flow of the paper is not very cohesive and somewhat difficult to follow. For instance, it was hard to grasp the key concept of main idea only reading Introduction. The initial introduction of understandable mechanism of the main method is introduced somewhat too late, which made reading before subsection 3.2 a little exhausting. I cautiously suggest adopting a “method-first, explanation-later” structure for the Method section, or providing brief mechanical information about KVDA in the preceding sections. Some sentences were too lengthy or colloquial. Figures and Tables of experimental results were confusing due to their jumpy locations and several formatting errors. Experimental conditions for each result were also confusing (especially the differences between Table 2, 6, 7 and Figure 4. Why do they appear to show different results?). Refining the overall flow with more intuitive and concise writing would be needed.\n\n#### Minor Weaknesses\n- Equation 6, it appears that the sign of d was not considered.\n- Page 7, unnecessary three “Section 5”?\n- Page 13, 14, Table errors\n- Page 26, Image error\n- Reproducibility link is unavailable.\n- Figure 3 (Page 27), numbering error\n- Figure 4 (Page 16), legend “(ε_a, εt)-OGA” instead of “(ε_a, 0)-OGA”?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FFaiG3mAvn", "reviewer": "Reviewer_EN6Y", "summary": "This paper proposes Known Value Difference Abstractions (KVDA), a framework that extends the ASAP (Abstractions of State-Action Pairs) methodology for Monte Carlo Tree Search (MCTS) in deterministic environments. Unlike ASAP which groups only state-action pairs with identical Q-star values, KVDA deliberately groups pairs with different values if their difference can be inferred from immediate rewards. The authors implement KVDA-UCT (a modification of OGA-UCT) that tracks difference-adjusted values during aggregation and a generalized version (epsilon_t-KVDA) for stochastic settings. Experimental evaluation across 19 deterministic domains and their stochastic variants shows KVDA-UCT detects substantially more abstractions than OGA-UCT and typically matches or exceeds parameter-optimized (epsilon_a, 0)-OGA performance without introducing additional parameters.", "questions": "Stochastic Failure Modes: The paper attributes epsilon-t-KVDA's poor performance to faulty abstractions. Can you precisely characterize what type of faulty abstractions occur, and why the aggregation/representative-switching mechanism fails to correct for them under transition errors?\n\nImpact of Heuristics: The board game results rely on dense reward transformations via hand-engineered heuristics. How robust are the performance gains to different heuristic choices?\n\nMulti-Player Extension: The paper notes V-star is not unique in multi-player settings. Could the KVDA framework be adapted to work with different value concepts, such as Nash equilibrium values or empirical best-response values from self-play, to broaden its applicability?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Elegant and Sound Conceptual Contribution: The core insight relaxing value-equivalence to known value differences is a natural and well-motivated extension of ASAP. Figure 1 effectively demonstrates cases where ASAP fails but KVDA succeeds. The difference-accounted aggregation mechanism is theoretically sound, and the proof correctly establishes the exactness (lossless) guarantee.\n\nStrong Empirical Performance (Deterministic): The method is practical, as KVDA-UCT introduces no new parameters. It empirically detects significantly more abstractions than OGA-UCT. Crucially, it consistently matches or outperforms a parameter-optimized (epsilon_a, 0)-OGA in both raw performance and generalization, all while maintaining negligible runtime overhead.\n\nComprehensive Methodology and Reproducibility: The evaluation is thorough, covering multiple metrics (abstraction detection, optimized performance, generalization), various iteration budgets, and detailed domain descriptions. The authors also provide reproducible code with compilation details.", "weaknesses": "Poor Stochastic Performance: The method's advantages largely disappear in the approximate (stochastic) setting. epsilon-t-KVDA exhibits mediocre performance, sometimes significantly underperforming. The authors acknowledge this stems from \"faulty abstractions\" but offer it only as future work, substantially limiting the method's practical applicability.\n\nExperimental Design Lacks Critical Controls: The use of hand-engineered heuristics to create dense rewards for board games may artificially favor KVDA's reward-difference mechanism. Furthermore, some domains show identical performance across all methods and some comparisons have overlapping confidence intervals, yet are still presented as favorable to KVDA.\n\nSignificantly Restricted Scope: The paper explicitly states the method does not extend to multi-player games due to the lack of a unique V-star. It also appears dependent on deterministic immediate rewards (not addressing stochastic rewards). These constraints sharply limit the method's impact, as MCTS is highly prevalent in the exact domains that are excluded.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)