Generalized Random Forests Using Fixed-Point Trees
Authors: David Fleischer
David A. Stephens
Archer Y. Yang
Venue: ICML 2025 spotlightposter (2025)
Keywords: Generalized Random Forests
Fixed-Point Methods
Ensemble Methods
Causal Inference
Computational Efficiency
Primary area: general_machine_learning->supervised_learning
TL;DR: We introduce a fixed-point tree-growing algorithm that replaces the gradient-based splitting mechanism in GRFs, enabling faster and memory-efficient forest construction while preserving estimator accuracy.

Abstract: We propose a computationally efficient alternative to generalized random forests (GRFs) for estimating heterogeneous effects in large dimensions. While GRFs rely on a gradient-based splitting criterion, which in large dimensions is computationally expensive and unstable, our method introduces a fixed-point approximation that eliminates the need for Jacobian estimation. This gradient-free approach preserves GRF’s theoretical guarantees of consistency and asymptotic normality while significantly improving computational efficiency. We demonstrate that our method achieves a speedup of multiple times over standard GRFs without compromising statistical accuracy. Experiments on both simulated and real-world data validate our approach. Our findings suggest that the proposed method is a scalable alternative for localized effect estimation in machine learning and causal inference applications.

Generalized Random Forests using Fixed- PPint Trees
David Fleischer" David A. Stephens
Archer Y. Yang 12
Abstract
1923; Rubin, 1974). The double machine learning frame- work (Chernozhukov al., 2018) unifies various ML-based causal estimation methods, including lasso (Belloni et al., 2017), random forests (Athey et al., 2019; Cevid et al., 2022), boosting (Powers et al., 2018), deep learning (Johans- son et al., 2016; Shalit et al., 2017), and general-purpose algorithogs (Nie Wager, 2021; Künnzll et al., 2019), 10,9, all of which focus on capturing variation over feature space.
We propose a computationally efficient alterna- tive generalized random forests (GRFs) for esti- mating heterogeneous effects in large dimensions. While GRFs rely on gradient-based splitting criterion, which in large dimensions is computa- tionally expensive and unstable, our method intro- duces fixed-pe lixed-point approximation that eliminates the need for Jacobian estimation. This gradient- free approach preserves GRF's theoretical guar- antees of consistency and asymptotic normality while significantly improving computational effi- ciency. We demonstrate that our method achieves a speedup of multiple times over standard GRFs without compromising statistical accuracy. Ex- periments on both simulated and real-world data validate our approach. Our findings suggest that the proposed method is scalable alternative for localized effect estimation in machine learning and causal inference applications.
Generalized random forests (GRFs) (Athey et al., 2019; Wa- ger & Athey, 2018) have emerged as a powerful tool for such tasks, leveraging adaptive partitioning with problem- specific moment conditions instead of standard loss-based splits. GRTs apply broadly to a wide range of important statistical models local linear regression (Friedberg et al., 2020), survival analysis and missing data problems (Cui et al., 2023), nonparametric quantile regression, heteroge- neous treatment effect estimation, and nonlinear instrumen- tal variables regression (Athey & Imbens, 2016; Athey et al., 2019). Unlike local linear models (Fan et al., 1995; Fan & Gijbels, 1996; Friedberg et al., 2020) or kernel-based models (Staniswalis, 1989; Severini Staniswalis, 1994; Lewbel, 2007: Speckman, 1988; Robinson, 1988) which suffer from the curse of dimensionality (Robins & Ritov, 1997), the tree-based approach of GRF offers more scalable solution.
1. Introduction
In many real-world machine learning (MI.) applications, practitioners seek estimate how quantities of interest vary across different feature subgroups rather than assuming uni- form effects. For example, medical interventions and policy treatments often have heterogeneous impacts across subpop- ulations, making localized estimation crucial for improving outcomes (Imai & Ratkovic, 2013; Knaus et al., 2021; Mur- doch al., 2019; Lee et al., 2020). Similarly, individualized recommendation systems adapt to user-specific features to enhance performance (Kohavi et al., 2013).
However, GRFs' gradient-based approach (Athey et al., 2019) becomes computationally expensive and unstable in large dimensions due to the reliance on Jacobian estimators for tree splitting. To address this, we propose gradient- free approach based fixed-point iteration, eliminating the need for Jacobian estimation while retaining GRF's theoretical guarantees of consistency and asymptotic nor- mality. Our method significantly improves computational efficiency while maintaining statistical accuracy, achiev- ing significant speedups in experiments on simulated and real-world datasets.
A key example of localized estimation arises in causal infer- ence, where modern applications prioritize individualized treatment effects over average treatment effects (Neyman,
2. Background and Related Work
'Department of Mathematics and Statistics, McGill Uni- versity, Montreal, Canada "Mila Quebee AI Institute, Mon- treal, Quebee, Canada. Correspondence to: Archer Yang <ancher.yang& mogill.ca>.
Given data (X,, O,) E X' x O, GRF estimates target function 6* (æ), defined as the solution to an estimating equation of the form
Proceedings of the 4rnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
EoIx (V))())) X = x],
(1)

---

Generalized Random Forests using Fixed-Point Trees
for all x E X, where db is score function that identi- fies the true (0* (z)) as the root of (1), and v" (x) is an optional nuisance function. GRF can be understood perspective as approximating 8 (a: through a locally parametric 0* within small neighborhoods of test point z. Suppose L(x) C {X:)i= is a subset of training observations of the covariates found in a region around x E X' over which 8* (x) can be well-approximated by local parameter. Observations X; L(x) serve as lo- cal representatives for in estimating 0* (a)) such that, given sufficiently many training samples in a small enough neigh- borhood of x, an empirical version of (1) over X; E L(x) defines an estimator 0 (z)) that approaches 0" (z)x),
Athey al. (2019) present (5) as the definition of the GRF estimator, motivated in part by the mature analyses of lo- cal kernel methods (Newey, 1994) alongside more recent work on tree-based partitioning and estimating equations (Athey & Imbens, 2016; Zeileis & Hornik, 2007; Zeileis et al., 2008). The GRF algorithm for estimating 0 (z) can be summarized as a two-stage procedure. Stage I: Use trees to calculate weight functions (a(x) for any test observation x E X, measuring the relative importance of the i-th train- ing sample to estimating 0" (-) near x. Stage II: Given a test observation X. compute estimate (x)) of 6 (x) by solving the locally weighted empirical estimating equation (5).
from a nearest-
Our contribution improves the computational cost of Stage by introducing a more efficient procedure to train the trees. Training the forest is the most resource-intensive step of GRF, and the cost of each split in the existing approach scales quadratically with the dimension of (x).. We adopt gradient-free free splitting mechanism and significantly reduce both the time and memory demands of Stage I. Crucially, solving Stage II with weights (z) following our stream- lined Stage I produces an estimator ((z) that preserves the finite-sample performance and asymptotic guarantees of GRF
1(2 E M()) [(x)]
(BL()) PI(O)) E Earg min 0
(O.)
In GRF, the set of local representatives L(x) is determined by tree-based partitions which divide the input space into disjoint regions, or leaves. The training samples X, that fall in the same leaf as form the subset I(x)). However, single trees are known have high variance with respect to small changes in the training data (Amit & Geman, 1997; Breiman, 1 1996; 2001; Dietterich, 2000), leading to estimates (2) that do not generalize well to values of x that are not part of the training set. GRF improves its estimates by leveraging an estimating function that averages many estimating functions of the form (2). Specifically, let (Ix) denote the set of training covariates that fall in the same leaf as x, identified by tree trained on an independent subsample of the data, indexed by 6 = 1, I...., B. The GRF estimator is obtained by aggregating the individual estimating functions (2) across forest of B independently trained trees, i.e. the solution to the following forest- aveerged estimating equation:
3. Our Method
In this section we describe the details of our accelerated algorithm for GRF We closely follow the approach of Athey et al. (2019), and define ((x) as the solution to a locally weighted problem (5) with weighting functions (xxx)) of the form (4). The weight functions are induced by a collection of local subsets {Lo())} such that each subset Ls(x) is determined by the partition rules of a tree trained on subsample. The construction of each tree, in turn, is determined by recursive splits of the subsample based on splitting criterion designed to identify regions of X' that are homogeneous with respect to H (z). Therefore, to fully specify the weight functions o; (x), we must describe a feasible criterion for producing a split of X.
1
n Σ
(0()).((z) arg min 6,y
an (o()))oo.
B
where (z)( = (x)) [Z]
Define observational weights (x(x) that measure the relative frequency with which train- ing sample X; falls in the same leaf as xr averaged over B
3.1. The target tree-splitting criterion for Stage I
trees:
In GRF, the goal of Stage is to use recursive tree-based splits of the training data to induce a partition over the input space. Each split starts with a parent node P C X and results in child nodes C1, C2 C. X. defined by a binary, axis- aligned splitting rule of the form C1 (X, Xx,e S t} and C2 = (X; X > f). where & denotes candidate splitting feature/axis and t E IR the splitting threshold. For a parent l' and any child nodes C1, C2 of P. let (0p, PP) and (0c,,,,,) denote local solutions analogous to (2) defined
1 B
ai (x)
(x),
(4)
for i = I,...,n. n. Then, the solution (0(z), (xx)) to the aveest-averaged model (3) is equivalent to solving the fol- lowing locally weighted estimating equation
(x)) E arg min 0,v
Σ (a(
(5)
2

---

Generalized Random Forests using Fixed-Point Trees
over the samples in P and Cj respectively:
the result of taking a gradient step away from the parent estimate in the direction towards the true child solution dc;-
Pp) E arg min 0,2
Σ feXXEM
(6)
Σ Barad 0p nc, (EX,EC,)
E AT 0p (10)
Do,) Dr,)) E arg min 0,
Σ
Vo (O:)
(7)
where (0p, (r) is the local solution over the parent, Ap is any consistent estimator of the local Jacobian matrix V (n..))BlVap..
(eX;EC,)
for j = 1,2. A strategy to split P into two subsets of greater homogeneity with respect to ('()) is as follows: Find child nodes C1 and C2 such that the total deviation between the local solutions ic, and the target (X) is minimized, conditional on X É Cj. = 1,2. A natural measure of deviation is the squared-error loss,
X; E Pl. and { can be thought of as a term that selects f-subvector from (0, -vvector, e.g. if R and IR then 5 such that 0 E (0,v) is the rectangular diagonal matrix 5 = [Ix 0j. When the scoring function v/2 is continuously differentiable in (0, v), the Jacobian estimator Ap can be computed as
C2) = Σ P(X E(X P)
2=
Ap = V ((,.)) RP (EX,EP)
E
V (0p.pe (0:)
E [xxx 0c, /
1 =
Σ
V(nN (Oi).
(11)
nP [EXIEP)
such that the resulting split (Ci, corresponds to least-squares optimal solutions 8c, and icg However, enr(C,, C2) is intractable since (-) is unknown. GRF considers criterion that measures heterogeneity across pair of local solutions over candidate split
3.2. Limitations of gradieet-basee approximation
The use of the Jacobian estimator Ap in (10) introduces considerable computational challenges. First, each parent node P in every tree of the forest requires distinct Ap matrix, which imposes a significant computational burden when explicitly calculating Ap V0p.pp (O;) to determine @erad Second, if the local Jacobian E( &p (O,) X, E P] is ill-conditioned, then the resulting Ap estimator may be nearly singular. This instability can lead to highly variable gradient-based approximations 03 and highly variable splits of P. For example, consider the following varying-coefficient model for outcome Y; given regres- sors W. = (W......, W(,)) in the presence of mediating auxiliary covariates X;:
ACC1, C2)
TC, 2 nip.
|0c, 0cal|
(8)
where nc,, nC2,, and n denote the number of observations in C1, C2, and P, respectively. In particular, rather than minimizing err(Ci, C2), one can seek split of P such that the cross-split heterogeneity between òc, and dc, is maximized. Athey et al. (2019) observe that enr( (Cr, C2) and (CCi, C2) are coupled according to err(Ci, C2) K ((P) - E [A1 C2)] + o(r3), where >> 0 is a small radius term tied to the sampling variance, and (PP) does not depend on the split of P. That is, splits that maximize (C,, C2) which emphasize the heterogeneity of dc, across split will asymptotically minimize en((C,, C2), which aims to improve the homogeneity of dc, within a split.
E X, = x]== 17 WW,
(12)
where (-) is a nuisance intercept function and (x()) (07 (x)).... ....... 02 (x)) are the target coefficients. Models of the form (12) encompass time- or spatially-varying coef- ficient frameworks, where (X,, Y,, W,) represent the i-th sample associated with spatiotemporal values X,. Such models are particularly relevant in applications like hetero- gencous treatment effects; see Section 5 for more in-depth discussion. The local estimating function (Y, Wt), identifying (0*(x), v" (x:) through moment conditions as in (1), given by:
Although the criterion A(C1, C2) is computable, evaluating is very computationally expensive since it requires solving (7) to obtain lc,+ 0c, for all possible splits of P. and closed- form solutions for 8c, are generally not available except in special cases of 1b. Instead, GRF approximates the target A-criterion based on criterion of the form
grad (C1, C2) TCC, TCC2 n
Marad C2
(9)
where 02 denotes a gradient- based approximation of 0c,- Specifically, 0grped is a first-order approximation interpreted
[Y: WTO W-T-v) v) W, Y; - T7-W. 0 -
VY, W.): =
3

---

Generalized Random Forests using Fixed-Point Trees
Boxples: 5iit regressor
3.3. Fixed-point approximation
1.30
To address the limitations of gradient-based approxima- tions, propose gradient-free approach based on the form of single fixed-point iteration. Let Mc, (8, == "c, Ejix,,ec)) (0,) denote the empirical estimating function for the child solution (&c,: DC,) such that (7) is equivalently written as:
0.27
Medod
0.770 HRT
1
0.30
Median 250 replication
(Bc,:Dc,) Pc,) E min | (0, j= 1,2. (15)
0,0
Nettod
Under mild regularity conditions, (0,, is a Z- estimator that solves the estimating equation Vc, (0, (0,u) 0. Reformulating this equation as fixed-point problem, write:
Reg
Figure I: Splits values (top) and split variance (hottom), with 10th and 90th percentile bands, across correlations of Wa,1 and Wap,.
(0, (0.v) (
17
(16)
=1(0,,)
A necessary and sufficient condition for (dc,, Prc,) ) to be a solution of (15) is characterized by the fixed lxxe--point problem (fc,,C,)) ) f(oc, De,)) where f is as defined in (16). Iterative fixed- fixed-point methods (Picard, 1890; Lindelöf, 1894; Banach, 1922; Ryu & Boyd, 2016; Yang et al., 2021) solve such problems by considering update rule of the form
Consequently, the corresponding local Jacobian estimator is
Ap 1
Σ
(Y,
npp
((:XEP)
W.W.T W
W
Σ (LX,EP)
(13)
RP
(0t, (btt +- f(0,v).
(17)
The form of (17) inspires us to approximate the true child solution Ic, using a single fixed-point update taken from the parent solution 0p:
When the regressors are highly correlated, the summation over the W,W, W.W. block of the Ap matrix leads to nearly sin- gular values of Ap, resulting in an unstable matrix inverse Ap!. and therefore unstable values of Aarad and unstable splits. This issue becomes more pronounced the number of parent samples n. decreases, is the case at deeper levels of the tree. These challenges highlight the limitations of relying on Ap as part of an approximation for the child solutions Ocz.
05, :0 " V., (0p,0p)
= gT
Vpp.en (Oi),
(18)
nc, (xX,EC,)
where the product with E is interpreted similarly to its role in the gradient- approximation (10) and to express the update (17) solely terms of the target --quantity. We in- terpret 0z. as approximation of Ic, obtained by taking step from 0p in direction that reduces the magnitude of the local estimating function Vc,, Notably, the approxi- mation
As an illustration, consider a simple varying coefficient model with primary regressors Wt,,, W2 (0, 1), auxil- iary covariates X, Unif(0, 1), and outcomes Y, generated
0ge does not involve the Ap matrix, relying only on the scores 0 (O;) evaluated at the parent solutions. In general, removing the inverse Ap' provides computa- tional cost savings of O(K3). The corresponding splitting criterion, which uses the fixed-point approximations 0ger
Y 1(X; > 0.5) + W1,2 +4,
(14)
where & ~ (0, Figure! illustrates the distribution of 2000 Á arad Asrad-optimal binary splits ttradient-based tree stumps) fit over 1000 samples of the varying coefficient model (14), repeated over different regressor correlation levels Corr(W.,,, 2 {0.80, 0.81,..., 0.98, 0.99). It is clear that splits based on the Arad crite-citt exhibit high variability when the correlation between the regressors is large. In contrast, our proposed method, discussed in the next section, does not suffer from the same problem.
as substitutes for Ōc, is given by
A C2)
nc, nea n
PPT
geeT
(19)
Revisiting the varying coefficient example from Section 3.2, we see that splits based on fixed-poin point approximations A

---

Note that our method does not rely iterative fixed-point procedures at all. Instead, it uses only a single step of fixed-point approximation to simplify the pscudo-outcomes. These simplified pseudo-outcomes are then passed directly to a standard CART algorithm for splitting. The numeri- cal convergence of our method therefore relies solely on CART's established and well-known stability, not on fixed- point iteration. CART splits pseudo-outcomed putationally efficient. Given a parent node P, the value Generalized Random Forests using Fixed-Point Trees
are significantly more stable than those based on Óggas Specifically, Figure I illustrates that splits that maximize (CI C2) are more robust to ill-conditioning in the underlying local Jacobian V (l (O.) X; E Pl, as is the case for highly correlated regressors in the varying coefficient model (14), and leading to highly stable splits.
where an analogous equivalence holds for A in terms of the gradient-based pseudo-outcomes. We demonstrate in Lemma B.1 (in Appendix B.1.2) that maximizing the fixed- point criterion A FPP FCT, C2) is equivalent to maximizing the CART criterion (21), and extend this property to any À -stylee criterion induced by pseudo-outcomes that can be expressed as a split-independent linear transformation of the parent scores 40p,pp (Ot).
3.4. Pseudo-outcomes
Approximations Oc, of the form (10) and (18) offer an addi- tional benefit: they enable the Å-criteria of the form (9) and (19) to be efficiently optimized through single multivariate CART split. A CART split performed with respect to vector- valued responses Ps E RKK over a parent node P produces a split (Ci, C2) that minimizes the following least-squares criterion:
com-
Σ
lp:- -o,
II +
Σ
ll 1 (20)
(E:X,EC))
(1:X,CC))
P: = - -BBaaara (O.) does not depend on a candidate split C2) for any matrix B that is fixed with respect to the parent. This allows much of the computation required to maximize A SP A C2) to be done at the parent level, and in particular avoids re-calculating the approximations 0gg and g across the sequence of candidate splits. Once P is fixed and p.PT are computed, the value of Á C2)
where Pc, ne, [(ix,e,,) p. Equivalently, a CART split that minimizes (20) will maximize:
(21) The equivalence between the split that minimizes the least- squares CART criterion (20) and the split that maximizes (21) is shown in Appendix B.1.1. GRF performs its splits by adopting gradient- pseudo-outcomes, defined as
ne, ||cc, 112 + nca l|
for the first candidate split requires ((np) time, and the value for all other candidate splits of P are queried in (1) time. While gradient-basee pseudo-outcomes share this property, the use of fixed-point pseudo-outcomes eliminates the computational overhead and instability associated with estimating Ap, discussed in Section 3.2.
grad_
E² Ap' V.r.pp (Ot)
(22)
such that the gradient-basee based approximation garad in (10) is equivalently written:
We show in Lemma B.2 (Appendix B.1.3) that choosing different values of y does not change the outcome of the fixed-point splitting mechanism. Specifically, the optimal split identified by CART on pseudo-outcomed p! of the form (23) does not depend on 17. This can be heuristi- cally understood by studying how the criterion changes as function of the candidate splits. To illustrate, we consider a VCM model of the form (12) for bivariate re- gressors W,, univariate X, E [0, 1]. and scalar outcomes Yg. A detailed summary of the settings is found in Ap- pendix D. The sequence of valid candidate child nodes obtained by split over univariate X, can be parameter- ized through scalar
ra 0p
1
Σ
P
grad_ 0p + Po,
gred
nC, (KXXEC,)
In the case of fixed-point approximation, we define fixed- point pseudo-outcomes:
F2r = n V.p (ap.bp(O.). 17 + 0,
(23)
such that the fixed-point approximation 0:55 in (18) is equiv- alently written as
05 = 0p + nC; (I:);CC))
Σ
EPT = 0p + PO; -
Per
per
(24)
(1 {X, X; S t) and C2(1) = (X; X, > 1). Let A(I) = (1,,()(()))) denote the parameterized target criterion (8). and consider the behavior of (tt), Arrad and two fixed-point criteria A (t) and A (t) of the form (25) based on pseudo- outcomes with scale factors 1) = 1 and v7 1 1/v2, respec- tively. Figure 2 illustrates the different splitting criteria values plotted against the sequence of candidate splits. The visualization clearly shows that the criteria curves for A()), Aarad()), (l), and A (L)(()) with = 1 are all very close to one
Substitute the above form of : into the Á TP crittrrin (19) to equivalently express the criterion in terms of the F2T pseudo-outcomes:
EPT A C2) = T1C nC22
TPT PC,
PC2
EPT
1|2,
(25)
T
'The multivariate CART criterion uses sum of squares impu- rity measure, as in De'an ath (2002); Segal (1992).
5

---

Generalized Random Forests using Fixed-Point Trees
Criterion values A(C 1.C1 over candidate splits (C),C2) C = XX stdandd C, Ki :X =t)
To compute the final GRF estimates 0(x) for the target 6* (a:), we follow the standard GRF mechanism for Stage II. After the fixed-point trees are trained in Stage I, test observation 2o E X' is assigned to local leaves (x(x), indexed by trees E {1,..., B). Each leaf (x contains the training observations that fall into the same leaf as zo in tree b. Using these local leaves, the forest computes training weights ay (2o)) as in (4). The final estimate d (zo)) obtained by solving the locally weighted estimating equation (5).
n=1
0.4
Critrion volue
A 1=1/42
Importantly, as discussed in Section 2, solving for ((xo) in Stage independent of the specific mechanism used in Stage The only requirement is that Stage produces valid weights. This ensures that Stage remains a stan- dard weighted estimating equation, enabling the fixed-point tree algorithm to integrate seamlessly into GRF's two-stage framework. We refer to the complete algorithm for es- timating 8 (x: using fixed-point trees as GRF-FPT. By preserving Stage II of GRF, the GRF-FPT estimator 6(x) retains GRF's theoretical guarantees of consistency and asymptotic normality while offering a computationally effi- cient tree- beseebbilding method. Pseudocode for Stage II of the GRF-FPT algorithm is provided in Algorithm 3, located in Appendix C.3.
0.0
0.25
0.75
Threshold
Figure 2: Criterion values across candidate splits (Ci (t), (2)) over threshold & E [o, 1]. The location of the optimal split under cach criterion is given by the corresponding vertical line.
another. Critically, the fixed-point criterion with y = 1/v2, Le. A (t), although scaled differently, still identifies the same maximizing split as A FPT (t). This is because CART chooses a split based on a rank ordering of the criterion over all candidate splits. The absolute scale of the CART crite- rion does matter, and it is only criterion rankings over the candidates that determines the optimal split. Therefore, choosing different scalar does not change the outcome of the splitting process.
4. Theoretical Analysis
Based on the scale-invariance of our splitting criterion, we now detail the recursive procedure for growing our fixed- point trees pseudo-outcomes with 7 7=
In this section, we provide a theoretical foundation for the GRE-FPT estimator 0(x).. For Stage I, Proposition 4. estab- lishes an asymptotic equivalence between the FPT criterion and a weighted oracle criterion Av C2) in (27), while Lemma 4.2 demonstrates that the Specifications A.2 are met by forest based the Av-criterion whenever they met by forest based on the A-criterion. Assumptions A. and Specifications A.2 are the sufficient conditions for the consistency and asymptotic normality of 0(x) in (5), and thus are used to formally justify the FPT algorithm as a mechanism for specifying an estimator of 0" (x:).
The fixed-point tree algorithm. The entire fixed-point tree- bre-growing procedure recursively applies the following two steps on a given parent node P:
Labeling: Solve (6) over to obtain the parent esti- mate (0p, Dp). Compute the pseudo-outcomes:
p - 0op.ep (Ot),
(26)
Proposition 4.1. Suppose Assumptions A.l hold, and assume moreover Neyman orthogonal moment condi- tions (defined in Appendix A.4). Denote by = (up(i:X)eP) |X; -5-rrl zrll the radius of the parent P, where up denotes the center of mass over X; E P. Let (oo denote the 0-block of V(æp) in (37). Denote by lllly the weighted Euclidean norm || = [[Vo((p)))])
for all é such that X, P.
(ii) Regression: Maximize AF (C1, C2) by performing CART split on the pseudo-outcomes p se over
3.5. Estimates of ((x) for Stage II
The fixed-point tree algorithm generates a single tree-based partition of X. Repeating this process over subsamples of the training data yields a forest of trees, each specifying local leaf functions In(x), These leaf functions define the local weight functions (2) via (4), completing Stage of GRF The full fixed-point tree training algorithm is de-
V Vo (x (o)))oo Define the weighted oracle crite- rion Ay (C1, C2):
1C, nzz
0èc. 0cal
Av(Ci,
(27)
scribed in Algorithm 1, while Algorithm provides the pseudocode for the porest-wide Stage procedure.
Then, treating the split as fixed with - E nc,, nc, and
6

---

Generalized Random Forests using Tixed-Point Trees
sufficiently small r 0.
5. Applications
In this section, we explore applications of GRE-FPT for two related models: varying coefficient models and heteroge- neous treatment effects. We consider an outcome model of the form introduced in Section 3.2. For each observation, let Y; denote the observed outcome, W; (Wi,,,... W.,))
A (CI, C2) Av(Ci, op
r2,
TC
Lemma 4.2. Let T(A) denote tree whose splitring mech- anism seeks splits that maximize (CC,, (2) defined in (8), and let T T(Dv) denote a tree whose splitting mechanism seeks splits that maximize (v C2) defined in (27). Sup- pose Assumptions A.1 hold and assume moreover that (()) is tree that satisfies Specifications A.2. Then, T(Av)
K -K-dimensional regressor, and X, a set of mediating auxil- iary variables, such that
Y, = "XX,, + W, (X,)
(28)
satisfies Specifications A.2.
where (-) is a nuisance intercept function, 0" (*()) (0) (z), (0(x(())))) (x))² are the target effect functions local
For Stage II, Theorem 4.3 establishes the consistency of the GRE-FFT estimator (z))
to X; = x, under the assumptions Ele; X; = x| = 0 and
Eie;W; X, = x] =0
Theorem 4.3. Suppose that Assumptions A.I hold, and let (((z),;()) be estimates that solve (5) based on weights induced by a forest of trees grown under the fixed-poin tree algorithm satisfying Specifications A.2. Then, (0(z), ((x)) converges in probability to (8* (z),v*(x)
Varying coefficient models (VCM). Given regressors W; E R models of the form (28) can be characterized as vary- ing coefficient models (Hastie & Tibshirani. 1993). As discussed in Section 3.2, we must also assume that the re- gressors W; are conditionally exogenous given X, = x.
The proof of Theorem 4.3 follows directly from Theorem 3 of Athey et al. (2019), which, under Assumptions A. estab- lishes consistency for estimates (f(z), (x)) that solve (5) with weights from a forest that satisfies Specifications 1-5. Thanks to Lemma 4.2, these forest specifications must also apply forest grown under the FPT mechanism. Specifi- cations 1-3 collectively impose mild boundary conditions the splitting procedure. Meanwhile, Specification 4 requires that trees are trained on subsamples drawn without replace- ment (Biau et al., 2008; Scornet et al., 2015; Wager et al., 2014; Wager & Athey, 2018), and Specification 5 requires that trees must be grown using an additional subsample split- ting mechanism known honesty (Athey & Imbens, 2016; Biau, 2012; Denil et al., 2014). Appendix C.1 provides detailed explanation of the subsampling and honest sample splitting procedure.
Heterogeneous treatment effects (HTE). A special case of (28) arises within the Neyman- Rubin potential outcome framework, which models the causal effect of treatment on outcome (Neyman, 1923; Rubin, 1974). Here, 0* (1) (01 (z)..... (x)) represents heterogeneous treatment effects associated with K discrete treatment levels. Let T. E {L..... }} denote the observed treatment level for the i-th observation, and Y; the potential outcome that would have been observed if treatment level k had been applied. The regressors W; E {0, 1]K in (28) are interpreted as a vector of dummy variables indicating the observed treatment level, W., = 1(T R:). The auxiliary variables X; account for potential confounding effects. The conditional average treatment effect of treatment level &s E {2...., K} relative to the baseline level = is then defined as:
(x())= = E (k)) Y,(1) X, = x²
Finally, Theorem 4.4 establishes the asymptotic normality of the GRE-FPT estimator ((x):
where the baseline contrast is set to (xr) = 0.
Theorem 4.4. Under the conditions of Theorem 4.3, sup- pose moreover that Regularity Condition I holds, and that a forest is grown on subsamples of size s scal- ing as 8 = n2, where B satisfies Regularity Con- dition 2. Then, there exists a sequence on(x) such that (x) - 0 (x))//n(()) ~ N(O,1) and (x) = polylog(n/s)--gg s/s/,, where polylog(n/s) s) is a function that bounded away from 0 and increases at most polynomially with the log of the inverse sampling ratio log(n/s).
Under exogeneity of the regressors, the target effects ('x) in models (28) are identified by moment conditions (1) for scoring function (Angrist & Pischke, 2009; Athey et al., 2019)
Va, W.) [Y: W, --70-v) v) W. Y, - W, 0 --
The gradient-bused pseudo-outcomes (22) are computed as
pf graad AAp' (W. -Ap'(W.-Wp)) (11.))))53 (Y,-
-Y (W, Wp) 0p)
The proof of Theorem 4.4 is an immediate consequence of Theorem 5 of Athey et al. (2019). Theorems 4.3 and 4.4 demonstrate that the GRF-FPT estimator is able to meet key statistical guarantees.
(29)
where WP P and Y P are the local means of W, and Y; over the observations in P. Centering Y, - Yr and W; W
7

---

Generalized Random Forests using Fixed-Point Trees
removes the baseline effect of the mean üp on pared, and where Ap is given by (13) as:
is consistent for the original FPT child estimator 0ee as | - - op(1), meaning that this approximation does not after the asymptotic behavior of our estimator. These accelerations are particularly compelling when the dimension of 0* (x) is large and computational efficiency is critical, as in large-seale A/B testing with multiple concur- rent treatment arms or observational studies with numerous treatment levels (Kohavi al., 2013; Bakshy et al., 2014).
Ap 1
E (W, - WW) Tp) -WP. (30)
TP (s:X,E))
Computing o in (29) involves the OLS coefficients 0p from regressing Y,-Y, p on W, over the observations in P:
0p Ap 1
E
-
(31)
6. Simulations
TP
(B:XEP))
In this section, we perform empirical evaluations of the computational efficiency and estimation accuracy of the GRE-FPT method. We let GRF-FPT1 denote the FPT algo- rithm using the exact form of the FPT VCM/HTE pseudo- outcomes (32) and let GRF-FPT2 denote the acceler- ated FPT algorithm based on the form of the FPT pseudo- outcome approximation (33) in Section 5, We compare both implementations relative to GRF-grad under VCM and HTE designs. Implementation details and links to the reproducible code are found in Appendix C.4.
In comparison. Pi in (26) are computed as:
p = - 1 (Y, Ws),
-WW w)) Y, Pp-
(W: Pp-(W:
)r)
(32)
The relationship pf Ap' pf
reveals significant benefit of FPT pseudo-outcomes. The form of प elimi- nates the computational cost associated with the multipli- cation of Ap' leading to ((3) computational savings. Furthermore, the computation of 0p in (32) no longer re- quires solving for Ap' Therefore, we can further enhance computational efficiency by using an accelerated form of pseudo-outcome T. instead of p!
Settings. We follow the structural model in (28). The aux- iliary variables X; are drawn from the Gaussian copula with latent covariance matrix Σ, where [Σ] (0.3) Supporting experiments for multicollinearity in X; can be found in Appendix D.2. The outcomes Y; follow (28) with Gaussian noise e; ~ NN 1). For VCM experiments, re- gressors W, E R are sampled from N (0. I). For HTE experiments, W, E {0, 13. follows multinomial distribu- tion, W; X x Multinomial(1, (T (2), ...., k (x))), where T (x) is the probability of treatment level & (1, K). characterizing a variety of different location- specific dependence structures through the setting of TR (-). We set ' (<()) = 0 and vary the target effect functions (xr) and treatment probabilities TT( across different settings, fully detailed in Appendix C.4. Throughout our experi- ments we use subsampling ratio s/n 0.5. Supporting experiments under different subsample ratios found in Appendix D.2.
of FFT = - (W, Wp p) - Y - (W, - p)) Tppp)
(33)
where 0p is replaced by 8p in (32), which is defined as one-step gradient descent approximation of 0p taken from the origin:
E (W-WW))(, - P)) 0p = 7 NP (E:XEEP)
(34)
Here, y denotes the exact line search step size for the regres-
sion of Y Yp on W;- - W over P:
I1(W - WP) - -1r)11 7 |I(W- - WP)(W W TYY (()Y))) P
(35)
Results. The relative computational advantage of forests trained under GRF-FPT displayed in Figure 3, while Fig- ure (in Appendix D.3) summarizes the absolute fit times across the three methods. These data show that the FPT mechanism is able consistently offer relative advantage, observing speedups of up 3.5x faster than the gradient- based approach at the largest dimension K = 256. Figure 3 also shows increasing gains with increasing K and provides an empirical measurement of the theoretical scaling benefits discussed in Section Moreover, the absolute fit times in Figure 5 (in Appendix D.3) illustrate that our method consistently remains faster than GRF-grad, with no clear computational or algorithmic bottleneck as a function of either or K. Supporting experiments exploring the ef-
where W = =Wi ... Wap]" and Y = [V ... y with the notation W - Wp and Y - Y understood as row-wise centering.
The computational cost associated with 0p comparatively small because many of the products that appear in (34) and (35) are alrcady computed as part of p ee in (32). Mean- while, we show in Appendix B.3 that the approximation for the FPT child estimator:
1 == 0p + nc, (u:X,EC,)
1
Σ
EPT di
8

---

Generalized Random Forests using Fixed-Point Trees
Fit time speedup factor: GRF -arargggaaa----- FFPT (forests)
California housing data: GRF-FPT2 Sporist-sarynng effects log median
Varying coeficiietmode (VCM)
log Households
Nedian Housing Age
log Median Incore
FPT1
FPT2
3.5
3.0
2.5
2.0
10000
1.5
1.0
Freccro
Ser Inndaos
Sant Cancarcker
VCM Setting
3.5
3.0
00002 000000 3
2.5
2.0
1.5
1.0
log Population
log Census Black Bedrooms
log Census Black Reoms
3.5
+5
3.0
2.5
00000
2.0
1.5
Car Freecss
Sur Fracacd
San Brecee
256
64
256
Regressor dimension IK)
Figure 3: Speedup factor for GRE-FPT in comparison GRE- grad for VCM timing experiments.
Figure 4: Gcographicallly-arryin GRF-FPT2 estimates estt-atssss 0(z).
fects of sample sizes up to = 500, 000 are presented in Appendix D.2, while Figures 7 and 8 (in Appendix D.3) show that even when n is small, GRF-FPT still observes a noticeable gain relative to GRF-grad. Additional timing benchmarks for VCM experiments and all HTE experiments are discussed in Appendix D.3.
under GRF-FPT2, with qualitatively similar results shown in Figure 16 for GRF-FPTI and GRF-grad in Appendix F, rela-
Figure 4 shows clearly the boographically-deeeedee tionship between different housing features and housing prices. In major urban centers such as LA, San Francisco, and Sacramento, housing prices tend to decrease with an increasing number of households, and may reflect over- crowding in densely populated areas. In contrast, rural regions show the opposite trend: prices rise slightly when rural areas have a larger number of housing units. This suggests that, in sparsely populated rural areas, a modest increase in households makes these places more attractive and livable. Median income, however, consistently shows a positive effect on prices across nearly all of California, while population size tends to show a negative effect, highlighting broader state- wide pressures on housing affordability.
To assess estimation accuracy, we evaluate the mean squared error (MSE) of ((x) across 50 replications of the model and testing on separate set of 5, 000 observations. Figure 6 in Appendix 333 confirms that GRF-FPT T matches the accuracy of GRF-grad, while significantly reducing computation time. Further comparisons for both VCM and HTE settings are provided in Appendix D.3.
7. Real Data Application
Data. In this section we apply GRF-FPT to the analysis of geographically-varyin effects 0* (xx) on housing prices. The data, first appearing in Kelley Pace & Barry (1997), con- tains 20,640 observations of housing prices taken from the 1990 California census. Each observation corresponds to measurements aggregated over small geographical census block, and contains measurements of 9 variables: median housing value, longitude, latitude, median housing age, total rooms, total bedrooms, population, households, and median income. We employ a VCM design of the form (28) where Y, denotes the housing value, X, denote the spatial coor- dinates, and W,- (W......, Wss)) are the remaining six regressors. Details of the model and data transforma- tions used for the California housing analysis is found in Appendix F.
8. Conclusion
Our results demonstrate that the FPT algorithm offers substantial computational advantage over GRF-grad with comparable statistical accuracy, and highlights GRE-FPT as a powerful method for multi-dimeensonal estimation, partic- ularly when estimates of the target function must be learned from the data rather than observed directly. Future work may explore extensions to larger-scale problems and alter- native estimation tasks, as in unsupervised learning and structured prediction. Our findings position GRF-FPT as scalable and robust alternative for practitioners seeking efficient localized estimation.
Results. Table 7 summarizes the computational benefit of GRF-FPT applied to the California housing data. Figure 4 illustrates the six geographically-varying effect estimates
9

---

Generalized Random Forests using Fixed-Point Trees
Impact Statement
URL http::////mlll.oooo/p/prrss///// html.
This paper presents work whose goal to advance the field of machine learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Breiman, L. Bagging predictors. Machine Learning, 24: 123-140, 1996.
Breiman, L. Random forests. Machine Learning, 45:5-32, 2001.
Acknowledgments
Breiman, L.,, Friedman, J., Olshen, R. A., and Stone, C. J. Classification and Regression Trees. CRC, 1984. ISBN
This work was supported by Natural Sciences and Engineer- ing Research Council (NSERC) Discovery Grant (RGPIN-
9780412048411
2024-06780) and FRQNT Team Research Project Grant (FRQ-NT 327788).
Cevid, D., Michel, L., Năf. J., Bühlmann, P., and Mein- shausen, N. Distributional random forests: Hetero- geneity adjustment and multivariate distributional regres- sion. Journal of Machine Learning Research, 23(333):1- 79, 2022. URL http:tttp//// /mmrr///mrrr///r///// ammlr.ong wwwggpapprs//2// 21-0585.m ht.l.
References
Amit, Y. and Geman, D. Shape quantization and recognition with randomized trees. Neural Computation, 9(7):1545- 1588, 1997.
Chemoozukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, Double/debiased machine learning for treatment and structural param- eters. The Econometrics Journal, 21(1):C1-C63 01 2018. ISSN 1368-4221. doi: 10.1111/cetj URL https:///oi. org/10. 111 1111ecct...10/
Angrist, J. D. and Pischke, J.-S. Mostly harmless econo- metrics: An empiricist's companion. Princeton university press, 2009.
Athey, S. and Imbens, G. Recursive partitioning for het- erogeneous causal effects. Proceedings of the National Academy of Sciences, 113(27):7353-7360 2016.
Cui, Y., Kosorok, M. R., Sverdrup. E., Wager, S., and Zhu, R. Estimating heterogeneous treatment effects with right- censored data via causal survival forests. Journal of the Royal Statistical Sociery Series B: Statistical Methodol- egy, 85(2):179-211, 02 2023. ISSN 1369-7412. doi: 10. 10.1003/jrsssb//kac000. URL https: http://doo....ooooo
Athey, S., Tibshirani, J., and Wager, S. Generalized random forests. The Annals of Statistics, 47(2):1148 1178, 2019. doi: 10. 10.1214/18-AOSTTT doi doi.org/10. 121 A001-0099
URL https://
10. www.10///rrabb//kaktae.e
Bakshy, E., Eckles, D., and Bernstein, M. S. De- signing and deploying online field experiments. In Proceedings of the 23rd International Conference on World Wide Web, WWW '14, pp. 283-292, New York, NY, USA, 2014. Association for Computing Machinery. ISBN 9781450327442.
De ath, G. Multivariate regression trees: a new technique for modeling species-environment relationships. Ecology, 83(4):1 2002.
Denil, M., Matheson, D., and De Freitas, N. Narrowing the gap: Random forests theory and in practice. In Xing. E. P. and Jebara, T. (eds.), Proceedings of the 31 International Conference Machine Learning, vol- ume 32 of Proceedings of Machine Learning Research, pp. 665-673. Bejing, China, 22-24 Jun 2014. PMLR. URL ht https:////// httpt:tt/tooceeeeeeeeeeee hptppppeeeenngs.cceee mlr. pww.srrsssvv denil14 4. html.
doi: 10.1145/ 2566486.2567967 URL https: 1145/2566486.2550
Banach, S. Sur les opérations dans les ensembles abstraits et leur application aux équations intégrales. Fundamenta Mathematicae, 3:133-188 181, 1922.
Belloni, A., Chernozhukov, V., Fernandez- Val, I., and Hansen, C. Program evaluation and causal inference with high- digh-dimensional data. Econometrica, 85(1):233-29) 2017.
Dietterich, T. G. An experimental comparison of three meth- ods for constructing ensembles of decision trees: Bag- ging, boosting, and randomization. Machine Learning, 40:139-157, 2000.
Biau, G. Analysis of random forests model. The Journal of Machine Learning Research, 13(1):1063-1095 2012.
Fan, J. and Gijbels, I. Local Polynomial Modelling and Its Applications, volume 66 of Monographs on Statisties and Applied Probability. Chapman & Hall/CRC. London, 1996. doi: 10. 10.1120199780203747755.. URL
Biau, G., Devroye, L., and Lugosi, G. Consistency of random forests and other averaging classifiers. Journal of Machine Learning Research, 9666):2155-203) 2008.
https: / /tt////www..co www.::awoooofffaiiic.o.o www.books...//ooooooooooooooo..e mono/100 12t. http:/100.2201//772203499999
10

---

Generalized Random Forests using Fixed-Point Trees
Fan, J., Heckman, N. E., and Wand, M. P. Local polynomial kernel regression for generalized linear models and quasi- likelihood functions. Journal of the American Statistical Association, 90(429):141-150 1995.
Lee, Y., Veerubhotla, K., Jeong, M. H., and Lee, C. H. Deep learning in personalization of cardiovascular stents. Jour- nal of Cardiovascular Pharmacology and Therapeutics, 25(2):110-122) 2020.
Friedberg, R., Tibshirani, J., Athey, S., and Wager, S. Local linear forests. Journal of Computational and Graphical Statistics, 30(2):503-517, 2020
Lewbel, A. A local generalized method of moments estima- tor. Economics Letters, 94(1):124 12211224,812,- 2007.
Lindelöf, E. Sur l'application de la méthode des approxima- tions successives aux équations différentielles ordinaires du premier ordre. Comptes Rendus Hebdomadaires des Séances de l'Académie des Sciences, 116:454 457, 1894.
Friedman, J. Greedy function approximation: a gradient boosting machine. Annals of Statistics, pp. 1189-1232, 2001.
Hastie, T. and Tibshirani, R. Varying-coefficient models. Journal of the Royal Statistical Society. Series B (Method-
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., and Yu, B. Definitions, methods, and applications in inter- pretable machine learning. Proceedings of the National Academy of Sciences, 116(44):22071 122000000 2019.
ological), 55(4)77577796, 1993. ISSN 00359246. URL ht http: //w:://wwwww www. jstor. org/ westto..rg//taabe///33/ http://taaeeee222333
Newey, W. K. Kernel estimation of partial means and a general variance estimator. Econometric Theory, 10(2): 1-21, 1994.
Imai, K. and Ratkovic, M. Estimating treatment effect
hetcrogeneity in randomized program evaluation. The Annals of Applied Statistics, 7(1):443 470, 2013. doi: 10. 10.1214/1-A0AS592, http: htwp/oorg//.... 121 2104/112-0A59..
Neyman, J. Sur les applications de la théoric des prob- abilités aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych, 10(1):1-51, 1923. Reprinted and translated in Neyman, J. (1990). Statistical Science, 5(4), 463-480.
Johansson, F., Shalit, U., and Sontag, D. Learning represen- tations for counterfactual inference. In International Con- ference on Machine Learning, pp. 3020-3029. PMLR, 2016.
Nie, X. and Wager, S. Quasi-oracle estimation of hetero- geneous treatment effects. Biometrika, 108(2):299-3) 2021.
Kelley Pace, R. and Barry, R. Sparse spatial au- toregressions. Statistics & Probability Letters, 33(3):291-297, 1997. ISSN 0167-7152. doi:
Picard, É. Mémoire sur la théorie des équations aux dérivées partielles et la méthode des approximations successives. Journal de Mathématiques Pures et Appliquées, 6: 145- 210, 1890.
URL sttps://ww..sceenediieeet com/ ocience/arriccll/ii///1111121121
Knaus, M. C., Lechner, M., and Strittmatter, A. Machine learning estimation of heterogeneous causal effects: Em- pirical monte carlo evidence. The Econometrics Journal, 24(1):134-161, 2021. 2021.
Powers, S., Qian, J., Jung. K., Schuler, A., Shah, N. H., Hastie, T., and Tibshirani, R. Some methods for hetero- geneous treatment effect estimation in high dimensions. Statistics in Medicine, 37(11::1777-1887 2018.
Kohavi, R., Deng, A., Frasca, B., Walker, T., Xu, Y., and Pohlmann, N. Online controlled experiments at large scale. In Proceedings of the 19th ACM SIGKDD International Conference Knowledge Dis- covery and Data Mining, KDD '13, pp. 1168-1176, New York, NY, USA, 2013. Association for Comput- ing Machinery. ISBN 9781 4503311747. doi: 10.1
Robins, M. and Ritov, Toward curse of dimen- sionality appropriate (coda) asymptotic theory for semi- parametric models. Statistics in Medicine, 16(1-3):285- 319, 1997. doi: 10. 101100////11)1107700038881190999 3(285:AID-SIMM SDDD-SSSS333..0.000222 pubmed ncbi n.m. nih. g
URL https://
2487575.2488217. URL https: //t.i///i///// //////iooog//../////////// 2145/2487575.2488
Robinson, P. M. Root-n-consistent semiparametric regres- sion. Econometrica: Journal of the Econometric Society, pp. 931-954, 1988.
Künzel, S. R., Sekhon, J. S., Bickel, J., and Yu, B. Met- alearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the National Academy of Sciences, 1 116(10):456-4144 2019.
Rubin, D. B. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Edu- cational Psychology, 66(5),688, 1974.

---

Generalized Random Forests using Fixed-Point Trees
Ryu, E. K. and Boyd, S. A primer monotone operator methods (survey). Applied and Computational Mathe- matics, 15(1):3-43, 2016. Survey article.
Zeileis, A. and Hornik, K. Generalized m-fluctuation tests for parameter instability. Statistica Neerlandica, 61): 488-508, 2007.
Scornet, E., Biau, G., and Vert, J.-P. Consisteney of random forests. The Annals of Staristics, 43(4):1716 1741, 2015. doi: 10. 10.1214/15-0SS1321, URL https:/ doi. wrg/10../....ooooooooooo. 1214/15-A05121..
Zeileis, A., Hothorn, T., and Hornik, Model-based recur- sive partitioning. Journal of Computational and Graphi- cal Statistics, 17(2):492-514 2008.
Segal, M. R. Tree-structured methods for longitudinal data. Journal of the American Statistical Association, 87(418):
407-418, 1992.
Severini, T. A. and Staniswalis, G. Quasi-likelihood esti- mation semiparametric models. Journal of the Ameri- can Statistical Association, 89(426):501-513 1994.
Shalit, U., Johansson, F. D., and Sontag, D. Estimating individual treatment effect: generalization bounds and algorithms. Precup. and Teh, Y. W. (eds.), Proceed- ings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learn- ing Research, pp. 3076-3085 PMLR, 06-10 Aug 2017. URL wwtptttpp::.. /tt::://prrroe ppw.eedinegggsssoeeeeeeeppppp ml press/v7D/ shalit175 7a. html.
Speckman, P. Kernel smoothing in partial linear models. Journal of the Royal Statistical Society. Series B (Method- ological), 50(3):413-436, 1988. ISSN 00359246. URL http:// //wt:://www..// /wtptr.org/stablle/3355555555
Staniswalis, G. The kernel estimate of regression func- tion in likelihood- basee models. Journal of the American Statistical Association, 84(405):276-255 1989.
Tibshirani, J., Athey, S., Sverdrup, E., and Wager, S. grfi Generalized Random Forests, 2024. URL https:// gi titb..... com gr alss gr.. R package version 2.4.0.
Wager, S. and Athey, S. Estimation and inference hetero- geneous treatment effects using random forests. Journal of the American Statistical Association, 113(523):1228- 1242, 2018.
Wager, S. and Walther, G. Adaptive concentration re- gression trees, with application random forests. arXiv preprint I 06388, 2015.
Wager, S., Hastie, T., and Efron, B. Confidence intervals for random forests: The jackknife and the infinitesimal jackknife. Journal of Machine Learning Research, (1):1625-1651, 2014. URL https: //tps:////m/rr/////////eee org/ papers/volume1 www.wllrllloooo 4a wwwerwlgllloooo 4a pdf.
Yang, Y., Gu, Y., Zhao, Y., and Fan, J. Flexible regularized estimating equations: Some new perspectives. 2021. URL https://arxtt wrggabs///e 11074.
12

---

Generalized Random Forests using Fixed-Point Trees
A. Technical Preliminaries
A.I. Assumptions
We follow the key assumptions of Athey et al. (2019) made for the theoretical analyses of GRF The predictor and parameter spaces are both subsets of Euclidean space such that x X [0, 1]" and (8,v) B c RK where
compact subset of IR.K Under the analyses of Wager & Walther (2015), we suppose that the features of the auxiliary covariates X; (X,,,..... p.p)) T have density fx that is bounded away from O and co, i.e. S fx (ax) S C < oc, for some constants 0 and C' < GRF does not require that the score function & is continuous in (0, v), as is the case for quantile estimation, one does require that the expected score/moment function
(z,((x) = Eo (O) | X = =x],
(36)
smoothly varying in parameters v).
ASSUMPTION 1. For fixed (0, v), the M-function (36) is Lipschitz continuous in
ASSUMPTION 2. For fixed æ, the M-function is twice-differentiable in (0,2) with uniformly bounded second derivative,
(o.u))) o0,
where |I-II denotes the appropriate tensor norm for the second derivative of MA. taken with respect to (0, v). Let denote the population Jacobian at the true (0 (x), 2/" (x)). and assume that V(x)
V(x) = (z) (==-(())-===(()) (
invertible for all x X. We write (ax) in block form as
V(x)= = Voo(x) (x)( Vuo(z) (x)(
(37)
ASSUMPTION 3. The score functions (O;) have a continuous covariance structure in the following sense: Let ((,.) denote the worst-case variogram:
[0
02 [/2
sup {|| SE
Varoix (W,)),((,) (0e)))(01) X; = x)
7
p1
then, for some L > O,
(
[0 F1
T22
0 F1
00
7
L
for all (11, (02, 22).
12
12
ASSUMPTION 4. The score function (O,) can be written
(O,) = X(0, v: 0,) + ((()))
where À is Lipschitz-continuuus in v), g: {0,) - IR univariate summary of the observables O,, and Co : + R any family of monotone and bounded functions.
ASSUMPTION 5. For any weights as with Σ 0; 1, the minimizer (0,0) of the weighted empirical estimation problem
(5) satisfics:
a (O:)
max fas), Sst
for 2 0.
ASSUMPTION 6. The score function (O,) is a negative subgradient of a convex function, and the moment function (X,, is the negative gradient of a strongly convex function.
13

---

Generalized Random Forests using Fixed-Point Trees
A.2. Forest specifications
The consistency and asymptotic normality results, Theorems 4.3 and 4.4, require that the forest trained following Algorithm consists of trees that satisfy certain set of specifications. These forest specifications precisely those imposed by Athey et al. (2019) for forests of gradient- based trees, and collectively, these specifications describe fairly mild conditions on the tree splitting mechanism, as well as specific requirements for the sampling procedure.
SPECIFICATION 1. (Symmetric) Tree estimates are invariant permutations of the training indices. In other words, the output of tree does not depend on the order in which the training samples indexed.
SPECIFICATION (Balanced(u-regular)) The proportion of parent observations assigned into either child is bound below by some w > 0, i.e. nc; 2 wnp.
SPECIFICATION 3. (Randomizeddrandommspttt) The probability of splitting along any feature/diimension of the input space is bound below by some > 0.
SPECIFICATION (Subsampling) Trees are trained on subsample of size s, drawn without replacement from n training samples, where s/n 0 as 8 +
SPECIFICATION (Honesty) Trees are trained using the sample splitting procedure described in Appendix C.I.
A.3. Regularity conditions
REGULARITY CONDITION 1. Let (ax) be defined in Assumption 2 and let p (z) denote the influence function of the i-th observation with respect to the target 0* (x:):
(x()) == -5 -o
Then,
Var(p; XX x) 0,
for all x e X.
REGULARITY CONDITION Trees grown on subsamples of size scaling n,, for some subsample scaling exponent / bound according 8min B 1, such that
log (w-1) log ((1 w) 11
-1
Amin :
B 1
where < n. w are constants defined in forest Specifications 2 and 3.
A.4. Neyman orthogonality
To identify the underlying local parameters (z), E F one must have a score (O) with at least K = Ka+K, components, where here we use K₂ and K2, to denote the dimensions of the component subvectors (x()) Ko and v"(x) E K. Conceptually, a score (O) can be partitioned into the components that identify the f-coordinates, denoted by w/, and those that identify the v-coordinates, denoted by 1/2. and thus the moment functions Ma, in (36) can also be partitioned the same way:
(0, (0,000
(x) = (1 [5:)]]
EL (0 X x EL v;
(O) =
M2(0, v;,r
=
X =
The corresponding Jacobian matrix of (x) taken with respect v) and evaluated the truth (0*(x), v² (x)) is
Voo(x) Voo(x)
Vor
V(x)= = ((.)) M(0, v; x (000
(z)
14

---

Generalized Random Forests using Fixed-Point Trees
where here the subscripts in the block expressions of V(x) indicate the coordinates with which the gradient is taken, and in all cases are evaluated at the truth (#* (x), et)), (x)):
Voo(x) Ve M1 v; (0) (s)
VoL (x) = V. Mi (6, 1/ 0= 1= (x),
V20(1) V M2(0, v; 0=0 00)) (4))
(x) = V. M2 v/ (s))
In this context, the assumption of Neyman orthogonal moment conditions is more completely labeled as Neyman orthog- onality for the estimation of (xx) with respect to the nuisance v² (x)) and can be summarized as an assumption that the moment conditions for 8* (z) are insensitive first-order changes in v around the truth v v²(x) whenever
0" (x)).. For and in other words, the partial derivatives of the moment functions for 8* (a:) with respect to v/ zero at (x)) (x))):
GRF, this means that assumes (1) satisfies M, (x:), v/ (z));
(x) 0.
A.5. Example: Neyman orthogonality for VCM and HTE
Consider the VCM/HTE model with data (Y, XX) related according to
B[Y; X; = x] = 1- (x) + Wx)
such that, as discussed in Section 3.2, the score function vha,> that identifies the underlying (0" v0)) is
[Y W 0 w)W,
(Y, W.)
Y; W; 0
and the corresponding local Jacobian V(x) has block form
W.WT W
W
W.W. ; X; 21
E(WT |x, = x!] 1
((z) = E
x; 2
E[W; X; = 2]
Therefore, for Neyman orthogonality to hold one requires that B(W, X, = 0.
B. Derivations and Proofs
B.1. Proofs for Section 3.4
B.1.1. MULTIVARIATE CART CRITERIA
Let P. E RK be vector-valued responses associated with covariates X₁ P. A standard CART split (C1, C2) of P minimizes the conventional least-squares criterion:
Σ
|l: - Pc,
1c
11
Σ
I - 1
(38)
(IEC))
(i:X))
where Pc, == = E(exec,)) Pi is the local prediction over child node Cj. We verify that a split (C2, C2) minimizes (38) if and only if it maximizes
ne, ||pc, 21 + nc. 1
(39)
Proof. Each sum (38) can be expanded as
Σ
||
Pc; 11?
Σ
l|::- 1pi-PGy pc, 112 1(x1 E CC),
(i:X)C))
(EXEEP)
Σ
(11:112- -2. pc, +||0c, l²) 1(x; E Cj),
(EXEP))
= Σ llaal12. 1 C2) ne. |,,, 11
(.X.EP)
15

---

Generalized Random Forests using fxxed-Point Trees
Therefore, the least-squares criterion CART (38) is equivalently written
Σ Σ j=12 (E:
|pi - |p:-,c,11?= Σ 1=12
Σ (i:X;E))
|alll 1x, - -no, |l (Pcs)1?)
Σ
Σ
Ilaill 1(X)EC)) - -(nci ll + no, l1
5=1,2
(H:XEE))
E
|leill
1ol11-(nu4) (nci ll 112 + ncs lPccal").
(EX.EP)
The first term does not depend on the choice of split, and therefore the split that minimizes (38) is equivalent to the split that maximizes (39).
B.1.2. SPLITS VIA CART ON -OUTOEES
The following result is generalization to the claim made in Section 3.4 that CART split pseudo- p. p. will produce split that maximizes the Å#Fr-criterion, and sufficiently general to cover gradient-based pseudo-outcomes p and the corresponding Á ggrd-ccriterion
Lemma B.1. Suppose we can write
oc, a
Σ Pi:
P -Brb (O,);
(40)
TC₁ (6::EEC;)
where and B denote appropriately sized vectors and matrices whose values do not depend the candidate child node C₃- Under Assumptions A.I, ,, the split (C1, C2) that maximizes
A(C1, C2)
AIc, PO02 0c, 0c2
n
is exactly the split chosen by CART for vector-valued responses Ps over covariates X; P.
Proof of Lemma B.1. The scores (O.) satisfy subgradient conditions by Assumption 6, and therefore the parent
solutions (0p, &p) satisfy the first-order conditions
Σ fi:XEPP
Ver.er (0:) =0 0.
Hence,
0
Σ
Pap.ip (
(0i)
Σ
10p.üp (Ot) +
Σ
10p (Ot),
(iXXEP)
(MXiEC;)
(:XiEC()
=-B
Σ
Vor.pp (0:) +
Σ
V (0.)
(EX,EC))
(#:XEC()
Σ P
Pr. (i:X,E(s)
(:X,EC1)
Each in the previous expression is equivalently written as Σm (li=nc, =m= (0c, a). Hence,
Σ P Σ Pi:
(i:XEE(1)
(iX.EC())
ney (0c, - aa) + +nc, (0c, - -a:
RCC, Oc, + NC2 Oc.
16

---

Generalized Random Forests using Exid-Point Trees
Writing Pc; mdy Σ (KXX)E),) Pb has:
_
Pc; 0c, a,
+cc
nic 0c
TCc BC2
np
np
n122
(0c. BC2))
np
and
nc, nēs
TC
Pic, 1|2=
[c. Ac,
up
n
Applying analogous arguments with respect to C2 has the symmetric result:
nCa
|Pc,
nCnnC n
0c. Ōc2
up
Therefore,
11 n
2
ne
0c, 12
llāc, 11 + ncc, (1@call)) nc,
0c, Ocs
0c.
+
7pp
n
ni
0cs 0oa
12
n
= A(C1,
Based on the arguments in Appendix B.1.1, a split (C1, C2) maximizes ne, ||cc, 11 + nc. llcc, 1l if and only if a CART
split performed on the P: over P. That is, (CC1, C2) is precisely maximized by single CART split on -BBbāp.0p (0:)
fit over covariates X; E P, as desired.
1.3. SCALE INVARIANCE OF CART SPLITS
Lemma B.2 (Argmax equivalence of FPT criteria). The optimal split identified by CART on pseudo-outcomes p. of the form (23) does not depend on the scale factor 1), for any 1) + O.
Proof of Lemma B.2. Denote by Pi (4) FPT pseudo-outcomes based arbitrary scale factor v) ş 0 of the form (23);
Pi (n)
-1 Vōp (O.),
(41)
and let Jc, denote the child-leaf average evaluated at the parent solution (0p, Pp):
v., nC, (E:XEC;))
Σ
such that the corresponding child-leaf pseudo-outcome averages PC, no, Eu.xec, Pi (n) are equivalently written
(n) 1.
pe. -n vC,
Let FFT (C1, denote the FPT criterion the form (25) based on pseudo-outcomes (41);
rer n
(C1, Cz) =
no: nC2
(n) 2 ne, nCs (n) - -CC,
|| (Va, - "
n
n
One has:
Inn ( - - =
|| -
(bc, ",
17

---

Generalized Random Forests using Txxed-Point Trees
and hence the A -criteria obey the scaling relation:
EET (2) - (CI, C2),
(42)
where A denotes the FPT criterion induced by gseudo-outcomen (1) based on unit scale factor 1. The relation (42) implies that any nonzero split-independent rescaling p. np, (1) Pi will induce a splitting criterion A (C1, C2) with the
same maximizer as A (C1, C2):
A STT (C1, = arg max (C1.2)
{Tp A7 (CI, arg max (O1.Ca)
{ ATPT((C)))))
arg max (C1.Oa)
Intuitively, CART split is chosen by ranking the criterion values among the candidate splits and selecting the maximizing split (Ci, C2). Therefore, the FPT splitting mechanism is unaffected by the scale factor 7 used to specify fixed-point pseudo-outeomes (23). The absolute scale of the A A does matter when searching for the optimal split, and only the criterion rankings across the candidate splits determine the final partition.
B.2. Proofs for Section 4 Notation and definitions.
Let (p b, c) = op(max b, et), with an analogous abbreviation for Op(-).
-xpl denote For fixed parent node P. denote by the center of mass of the X, E P. and let r = (ux(ex || - xpll the radius the parent P. Throughout, we consider an asymptotic regime where nc, - oc and - 0, corresponding to leaves over of vanishing radius. Further, r and nc, are related under the conditions GRF Proposition 1, namely, nc, and hence ne, r2 oc and 1/ /nc, o(r).
Let 0G2 denote the true parameter expectation over the child node:
02 == (X) X E Cyl,
j j= 1,
1,
(43)
and let 0ūg (xp) denote an oracle version of the gradient-bas -based leaf statistic:
02 (xp) = (xP) nc, (B:XEEC))
Σ
where V(z) is the underlying local Jacobian in Assumption 2. Equivalently, in terms of the oracle pseudo- outcome/influence function (-) defined in Regularity Condition 1,
0e (1P) = ('(p)) + nc, (EX,EC;)
Σ
(Tpp)
The following technical lemmas used for the proof of Proposition 4.1.
Lemma B.3. Suppose Assumptions A.I and Specifications A.2 hold. Then,
ACC C2)= nc, nC2
|66 - 88 +op 2 nc.
np
RCa
Proof of Lemma B.3. Write the difference 8c, 06
oc, 0C2- -Oc, 0c, (xr) + 002 (xr) - EiOč, (xr) X X + (1(02((r))
X E Cy] 0c2
Ti
T
18

---

Generalized Random Forests using Fixed-Point Trees
Under standard LLN arguments, the second term satisfies T2 O1p(1/ (T,)) one has 72 (r)). Meanwhile, the first and third terms appear in the proofs of Propositions and of Athey et al. (2019). respectively, and satisfy Ti op(r, 1/ mC, and T3 O(r2) = T3 o(r). It follows
). and in an asymptotic regime with r-2 < nc;
c 0c oo (r, 1/ w/v,c,))
and in particular
dc, Ac, Pc;, ffc, + op r,
TCc
VTCa
Write = 8 - C2 and let E be any term satisfying F. = or(r, 1/1To, written AC11, C2) (nc, nc3// ||A + ElI Consider the difference
1/ TC,) such that A(C1, C2) is equivalently
A(11,C)) nc; nC2
1002 0c,1l² = ne, nC2
(IA + E1| -1|411
2
nip
2 TCC, RC2
((A,, E + |[E|))
np
Under Specification 2 there exists a fixed proportion w(1 - w) and also nc, /c3 & 1/4 for all nc, + nc2, = np. Therefore nc, /n3. = O(1) Meanwhile, || (r(r? 1/nc,, 1/nc,) is true by definition of E, and under our assumptions one may follow the arguments of Athey et al. (2019) Proposition to see that A 0c -00-- = Or)). Thus,
such that no1, nC2 2 wTp, and hence no, nc,
O(()) op vnci" =
Vncal
=op
(r2
Vnc,"
r
VTc,
and therefore
C2) ncC nCC2
f62zll 10 or
(,_ nc,
1
1
=
n
nC2
as desired.
Lemma B.4. Suppose the conditions of Lemma B.3 hold, and assume moreover Neyman orthogonal moment conditions such that the underlying Jacobian ((x) defined in Assumption 2 with block form (37). Then,
nC, n7² Voo(E))(( 00.)1l oop (?2 nc, ncz
A AEPT A C2)=
n
where FPT defined in Lemma B.2 denotes the FPT criterion with arbitrary scale factor 1) #
Proof of Lemma B.4. From the proof of Lemma B.2 one finds that A A C22) is equivalently written
A (C,, C2)
nc, nC,,
Σ n² T - E.)) JC, ne, (EX,EC,)
1
Ppr.o (O₂).
n
Under standard LLN arguments the average scores Ve, satisfy
JC2
Eleb op.ep x (00
(44)
One applies iterated expectation to see
Elwa,, (0) X CCC, C,
E dh Her.er (0) [0)
X
B[M 2p (X) CC).
and hence
cc,- E[M (x) X
(45)
19

---

Generalized Random Forests using Fixed-Point Trees
Expansion of M& (X). Under Assumption considers the Taylor expansion of Map Pr (X) about (0, (0 ( (τp)):
Ma Mpp (X) = (x)
O0p - (xp) üp v² (xp) +Op
(11e0- (xpp) D²
(a(()) (xp) 10)
Mo. (en)
The consistency of the parent solutions (0p, Pp) for v is established by Athey et al. (2019), and in particular 1/ (0p, ip)) (0 ( (zp)) Op(r, 1/vup). The asymptotic regime r < up implies 1/ /up o(r) and therefore the higher order quadratic term is equivalently expressed:
v²
0p 0p-00(xP) (xp) Dp V (xp)
Mar.er (X) (e-()))) + Mie.:
+Op(r²),
and therefore
Mor. (x) E C3} =E (oD Cj]
0p 0 (xp) @p - v (xp)
Me x E c,)
+ +p(r)))
One has ((o.s) M (o-(e))
((xp) -X))) because Ma is Lipschitz in x, and the expansion in the previous
display becomes:
00p 0p-00(2p) (zp)
E (X)
(X) X E Cj}
E (C)]
+ +Op(r?).
(46)
Up V (IP)
Expansion of Mo (wp) (X). (X). (0, v) (0 (X), v² (X)) as:
Following similar arguments, the term (x-( (X) is expanded about
M (an), Morr)) + (0'x)) - (X))
(((p))-'(() +Op(1²),
=V(X) (x)x)) - (xp))-0(()) v0 (xrx)) -x() + +pp2)))
where Mg-(X) (X) holds because (0" (X",),, (X)) defined satisfying the GRF moment conditions (1) local to X. One takes the conditional expectation of the previous display:
((rr)) v (xp))
x + +r((1²)
E [Mo (op) (X) X E CJC = E v(x)
Whenever X Cj one has ||X - xpl| (r). and the same Lipschitz arguments can be applied to see V(X) V(xp) + (r(r) conditional on X E Cy. and the previous display simplifies:
(xp) Op(r3), v (xp) vč
(er)) X E C, V(rp)
(47)
where 87 = E(0*(X) / X E CyJ and vē = E[v' X E Cý}. Substitute (47) into the conditional expectation (46):
0 (0)))) - 0% 1² (xx)) V
Op - (xp) Op 1² (IP) + +Op(r?),
E [M (x) X (X) = -(pp)
V(Ip)
0p 02
=V(pp)
+0p(12).
Up v2,
20

---

Generalized Random Forests using Fixed-Point Trees
Therefore, the child node score averages Ve, in (45) satisfy
0p 022 Up VO,
Vc, = (xpp)
+0p (r?, 1/vnc,)
and the difference vec,- cc., satisfies
Or - 02
0p -000
1
c. vc.- = (Zpp)
(xpp)
+0p
p?,
Op - V(2
[pp VC22
vnc,
vnco
[66 Cc2
1
1
+ +Op
Lec V222
Vncs
vnczs
We assume is a fixed scalar and 5 fixed matrix, it follows:
8/ 022 VC: 1122
mE (vC, - Tc,) - = - VV(p))
+ +0p
(48)
Vnc,
Vnc²
The fixed matrix < selects the coordinates of the target effect as & {T(0,
0, and hence the product & (xxp) simplifies:
((T()) = (oo(#p) (xp) [Voo(<p)
Vou(xp)]
(vr( (ar()))
Under Neyman orthogonality one has (xp) = O, implying that (Vx(x))= = =2Vo(<pp) 0), and (48) becomes
ne (0c, - -c,) = - (Voo(I))((E - -0,,) +p (1², / vnc.
1
(49)
Vno."
Asymptotic analysis. Let E be any term satisfying E = Op(r2, 1//nC, 1/ 1//)),). In our asymptotic regime with ne, = 1/ nCc = o(r), one has
E O= 2 1
v
E op (r. /nc,"
r.
1
1
Vnc,"
vnca
vnca
and therefore (49) satisfies
1
ns (Tc, - - -nVoo( - -00,) +op r,
(50)
VTc
Vica
Write A Ves (02, - 022) such that (Cr C2) equivalently written ( C2) (nc, /n²) 172 1|A E.1 Consider the difference
(CT, C2) no, 1122 n² Voo 00,))l2 n1,,1102 1 (| + E11² - -|A|11) 2(A, E) + +11B1²)
n²p
no, n0, "1
n
One repeats the same arguments used in the final asymptotic analysis of Lemma B.3 to show
2(A, E) + ||ا Op (?²
1
1
nes
noz
and thus
nc, nc, 1? |Voo - -fc,)112 op (
A FPT
(C1,C2) C2)
2
np
nCi
nCa
as desired.
21

---

Generalized Random Forests using Fixed-Point Trees
Proof of Proposition 4.1. First, under Assumptions A.I the 0-block (eo(#p) of the local Jacobian ((xp)) is strictly positive definite and thus ||-Ily defines true norm. From the proof of Lemma B.3:
c 0c 0c,
fcc f + op (",
TCC
The matrix V(rp) is non-random and fixed given P and 1) is fixed scalar. It follows:
(r,
(ooo &c,) n)Voo(=)) 0c,,) + +or
VnC.
VTC,
Up a negative factor, the expression on the right is precisely the same as (50) in the proof of Lemma B.4, and thus one repeats the arguments arrive at
[oVo - -cc2)( [[Vol(z)) - 0222)12 oo
(/2
nc,
nc2
and hence
(C1. TC, nc2 of (Voo(z)) 000))112 n²p
y2
nc,
nc²
The right hand side is precisely the same as in the statement of Lemma B.4 established for T,PT(Ci, C2), and thus
3 A C2) Anv(Ci, = =op ,2
1
1
(51)
nc
nC2
as desired.
Proof of Lemma 4.2. Firstly, Specifications 4 (subsampling) and 5 (honesty) describe conditions imposed on the sampling mechanism and are not affected by the form of the splitting criterion. It remains to verify whether Specification I (symmetry), Specification 2 (balanced/u-regular)) and Specification (randomized/random--piil are satisfied by T(Av).
1. Specification 1: Symmetry. A tree said to be symmetric its estimates are invariant under permutations of the tree's s training samples. Conditional on a sequence of criterion values computed over splits of P, the CART mechanism of selecting the best split by scanning over the collection of candidates does not depend on the parent samples at all. This means that asymmetry could only enter through the criterion values. Therefore, sufficient condition for symmetry in the tree estimates with respect to permutations of the tree samples is whether the criterion Av (C1, C2) is symmetrie. Conditional on the child solutions 0c,, Ôc,, the map (0c, dc,) + (v C2) does not depend on the parent samples at all, and therefore asymmetry could only enter through child solutions dcy- However, both criteria use precisely the same child solutions Oc, in (7), and therefore (C1, C2) will be symmetric whenever (CC1, C2) is symmetric (specifically, whenever o (O,) is symmetric with respect to permutations of i).
2. Specification 2: Balanced/w-regular: This condition is enforced by by GRF by adding additional stopping condition to the gradient-based version of Algorithm 1. Specifically, GRF stops a recursive splitting path if a proposed A-optimal split were to send fewer than w of the parent samples into either child. Simply stated, GRF enforces balanced splits by defining the set valid candidate splits as those with at least wnp parent samples each child. This is left unchanged by our method.
3. Specification 3: Randomized/random-pee
The asymptotic theory GRF requires that, each node, each variable selected for a split with some lower bound probability æ > 0. In order to satisfy the minimum split probability GRF uses the feature sampling device of Denil al. (2014) which, at each step, considers only min min(max (waa/mmaaissssaeee (Poisson(m). 1), p} randomly selected features candidate variables, where p dim(X') and is GRF tuning parameter. In other words, GRF defines the set of valid candidate splits such that the set of valid splitting dimensions is itself a random variable. This mechanism is left unchanged under our method.
No column of V (zp) is all-zero V. (xP) + 0 because V (xxp) is strictly positive definite symmetric, and therefore Av C2) will not systematically ignore signals along parameter dimensions 0g that can be detected by the A- criterion. Finally,
ic, dc, # = (oo(xr)(Oc, .c2,) # #0,
22

---

Generalized Random Forests using Fixed-Point Trees
because (x) is strictly positive definite symmetric by Assumption 2. Therefore A(C1, C2) 0 0 meaning that the Av-criterion is non-degenerate and will always be able to select least feature whenever the A-criterion can select feature.
Av(CI.C.)
Therefore, all five specifications are met, and one concludes that T(Av) must satisfy the forest Specifications A.2 whenever they satisfied by T(A).
B.3. Asymptotic equivalence of the pseudo-outeome approximation for VCM/HTE models
In this section we establish the asymptotic equivalence of the further acceleration of the fixed-point method proposed in Section 5 for VCM/HTE models. The accelerated algorithm is based on FPT pseudo-outcomes that use approximation Op for the actual parent solutions Op in (31). Specifically, the parent-leaf approximations Op are found by a single gradient descent step towards 0 taken from the origin (54). Let pf denote the original FPT pseudo-outcomes for VCM/HTE models:
Pi = (W, WP) (Y. Yp - (W, W Wp) T6p).
(52)
where the solution 0 for the local model over the parent P are precisely the OLS coefficients from the regression the centered outcomes Y; - y E R on the centered regressors W; - Wp IRK, contrast, let क: denote approximations of pf* pseudo-outcomes that are of the form
PTT - (W; - Wp) (Y,-Yp - (W, WP))Tp)
(53)
where p approximates 0p as:
0p nP (KX.EP)
Σ (W, -YP) =7. np
WT W.YYP.
(54)
Here, Wp Raex and Yp Rep denote the centered data matrices, Wr W W Wlexxer and Yp \ Y Pl and the scalar > 0 denotes the exact line search step size corresponding to the regression of the centered outcomes on the centered regressors:
IWTYP WP W Yp
(55)
Lemma B.5. Let 0c, denote the FPT estimator of the form (24) for the child solution cc, for VCM/HTE models. One can express dc, in terms of the corresponding fixed-point pseudo-outcomes.
0c, 0p 1
ΣE
JPT
P
TGG s:X.EC,
Similarly, denote by dc, the FPT estimator of 0c, induced by pseudo-outcomed approximations ,.PT:
0c, 6p nc, (i:X,EC;)
Σ
Then, under the assumptions of Proposition 4.1, Do, consistent for 0c, as:
0o. flc--cc,1 0c, 00.1|=0pp(] op(().
Proof. A direct calculation reveals that the difference between the original FPT pseudo-outcomes p.P in (52) and the approximations TFPT in (53) satisfy
TPT o. = -W, ([y:-y>>
- 11y;-Yp-- (W; -Vp 0pl - [y, (axp)=[vx--e -
- Yp
-W; (W)-W))T0p))
Pi
-WW,- (WW-W))(W, WP) ((p) -0)-
(56)
23

---

Generalized Random Forests using Fixed-Point Trees
Therefore, the difference between the original FPT child estimator Oc, and the approximation 0c, satisfies
0c, - -c, ne, (s:X,EC,)
1
Σ
(pi
SPT
d)²T),
1
E
(W; - Wp)(W, (p) 0p),
nCs (i:X,EC;))
= =Sc - 0p),
where we denote Sc, no,
1 E E(xx,Ec)) (W; W p)W W p)T. Therefore,
|0c 1 -c, = ||s (pp- 11:0-8p)) 8p) = $|ssc, l1 00 - 6p||
(57)
Under GRF's regularity conditions, in limit where nic and the parent radius r == (iX:e))))) || Xr| goes to zero + O, we have Sc, 3 Q for some positive semidefinite symmetric matrix Q, and hence |Sc, ll = Or(1).
Meanwhile, by definition (31) for the OLS coefficients 0p and definition (54) for the one-step approximations 0p, we have
0p 0p
||( (-App . wPr) - (n. || (-Ap' -T1) n 1p)' wp
Ap - -
1| w 1
- r [n]'
Inp' wep'l
(58)
- where -Ap = Rp WTT W. Wp follows from the definition of Ap as an estimator of the Jacobian Vel, e.g. (13) in the context of VCM/HTE models. Under the Lipschitz continuity Assumptions & 3, one has the standard stochastic bound for the cross term W,'W, Yp:
|n Fop',,Yrl1 0p
Op (r. /np/
}
1
(59)
while the difference [np' 1 S.WWWW, WW I is stochastically bound as
[W Or(1),
WT Wr 3s Cov(W, X; P) is non-singular under Assumption 2. Coupling these stochastic bounds together according to (58) gives
because '
| 0r =0r
npp
and trivially, because nc, TIP.
00p - 0p Op T.
(60)
TTC
Under Proposition GRF one assumes r - < nCC, and thus, an asymptotic regime where nc, - 3o and - 0,
one has 1/ /Tc, o(r), and hence:
10p 0p|| = (1).
(61)
Returning to (57), the consistency of the parent approximation 0p as (61) implies that the approximation 0c, is itself consistent for the original FPT child estimator dc.:
0ōc; 0c, op(1),
(62)
as desired.
24

---

Generalized Random Forests using Fixed-Point Trees
C. Implementation Details
C.1. Honest subsampling
In this section we present the honest subsampling mechanism. Trees are used to form partitions of the input space such as to to specify weight functions a: (z), defined as
(1(x) Σ (b)()), for (xi(x) 1(X, E Lb(z)) [Lx(z)
1 n2
(63)
where Lx(r) denotes subset training samples that fall alongside according to the partition of tree 6. The honesty mechanism ensures that observation in leaf (z(x) was used to build the partition rules of tree b. This is achieved by separating an initial subsample into two subsets: One for building the partition rules, and the other allocated as samples to the local leaves (z) according to the trained rules. Below, we give a detailed outline of how subsampling and honest sample splitting is used to train a forest of trees, then show that weight function a; (z) given by honest trees according to (63) conditionally independent of O given X,.
Honest subsampling for GRF For E Bl.....B.
1. (Subsampling). Draw an initial subsample (()) of size [Z( from the training set (without replacement).
2. (Honest splitring). Split (b) into disjoint sets J(th and J.2 of size (T(h) = -[s/2] and (T,2 [s/2].
(a) Train tree ((J(")) based on the first subsample {(X,, O,) i J( Let R( ( denote the partition of X' induced by T(J(()) such that
R (h)
{ x X' satisfies the partition rules for leaf m T T(J
(b) Subset the samples from the second subsample {X; : E J. according to the trained rules of ((T(")). i.e. the samples of T( in the leaves determined by the rules of (T)
For any x E ,, the local leaf (x) that appears in (63) defined as the specific subset of Jh"h) samples belonging to the same leaf of tree (() x. Lo(x)= = (X₂ R(0) :i J. and E R(O)
Conditional independence of (z) and O given X,, By definition, the partition rules of tree (()( depend only on the ((()) subsample. The rules of tree operate only on covariate values, and therefore the task of subsetting (X; : : J. into leaves according the rules of T(](") requires knowledge of the X; values from the J. subsample but not necessarily the O,, Based on this understanding, we will show that (x) is conditionally independent of O given X, Based on (63), it is sufficient to show
E ((ona(x) O; Xi] = B[(oa()) X.].
Case 1. Suppose i & [((O. By definition (1) c (X j E (b)) is immediate that 11 ((X, E [T(x))) = 0. and therefore agg (n)(x) = 0, and trivially
(())(()) X,] = (aa)((r)
X,] O, for all i J.u)
Case 2. Suppose E J,(6) We show that each component used to specify one (x) in (63) conditionally independent of O given X
Tree ((J(")) is trained using only the J( subsample. This does not depend on Og, for all i E J. conditionally on X; or otherwise.
25

---

Generalized Random Forests using Fixed-Point Trees
The rules of tree ") operate only on input values. Therefore, conditionally on X; for all i Tg the leaves of the Tz subsample specified by tree T(){()) do not depend on the value of O
Leaf [s(x) is the specific subset of the Tz samples satisfying the same partition rules of T(T(()) as x. Given the leaves have been specified by the previous step, this depends only on a.
Therefore, the individual component functions one (z) 1((x; E Lo(z))))\Lo(zr) are conditionally independent of O, given X,,
B(au;()) | X. = [[as:()) Xi),
for all i Jz
Demonstration of honest subsampling. Let ((X,, denote training set of 20 observations, where each
X, = (1,2, is over X' IR2. We will use forest of single tree (B = 1) to specify the functional form of weights (;(x).
1. (Subsaamllng). Draw an initial subsample I of size 8 = 10.
2. (Honest splitting). Split I into two disjoint sets
Ji and J2 each with s/2 samples.
Ti
X1,1
X1,2
O₂
i E
I
X1.1
X
O
2
2
3
3
J1
8
5
15
Xa.1 X1,2
8
20
10
20
Ta
X
X
O₂
5
1
J2
10
2
-2
11
0
14
1
-2
16
2
(a) Train using the data from the first subsample has the following structure:
J1 inducing partition of X'
RR2 Suppose the fitted tree
X
X,, <
R2
yes
no
Na
-1
1
2
X5,2 < -
R3
R
yes
no
R1
R
R2
2
(b) Use the trained partition rules subset the Ja subsample into separate leaves.
26

---

Generalized Random Forests using fxxdd-Piint Trees
X1,2
X16
X11
R2
X5
X11
R3
R
X
X10
The tree trained on the Jr subsample will subset the
J2
subsample as
{X; E [2] (X10, X14) (Xs, X11, X16) U0
where include the trivial union with 0 to note that the tree assigns none of the T2 samples the partition of R²2 where X,, 2 3.
The leaf Lx(x) is the specific subset of the T2 subsample such that X; E Th satisfy the same partition rules as x. Given a test point To, there are three possible scenarios for (s(x)) that correspond to the three regions R1, R2, Rg C IR in which the test point zo can appear.
Region 1.
If Fo E R1 then Is(za) (X10, X14) and
amx(oo) = 1([X, E Lo(zo)t)
if e 14), otherwise.
[La(xo]]
0
Therefore, (x(x0)= 3 for i 10, 14 and zero for i {I,...,20) {10, 14).
Region 2. If Zg E R2 then Lo (zo)) (15,X11. X1. and
abi(zo) (((x E Is(mo)))
1 if (5, 16,
3
[Lo(zu)
otherwise.
Therefore, (x())) for i 5, 11, 16 and zero i E
\ {5, 11, 16).
Region 3. If 20 E R. then Ta (x()) 0. This a degenerate case such that
Orsi (wo) 11{x; Lo(zo)))
[Tx(2))
undefined, leading to a non-identifiability problem whenever xo E R3. When this occurs, Tibshirani et al. (2024) recommends calculating (z()) based on only the trees with non-empty Lx(x)). Let (b fI...., B} [Lo(2o) > 0} denote the indices of non-empty leaves associated with zo. Then, the GRF weight functions based on this recommendation be written
(x(20) =
Ciba (wg).
BeB
27

---

Generalized Random Forests using Fixed-Point Trees
Algorithm The fixed tree algorithm
function TRAINFIXEDPOINTTREE
Input: node .V
node Po <- GETSAMPLES(N)
queue Q < INTTIALIZEQURUE(()))
while NoTNuLL(mode P POP(Q)) do
(0r, 2r) <- SOL.VESSTTIAATTIGROUUUTTOON
Computes (6).
p p --FTTPSUUDDOUUTCOMMSs(.... up)
> Applies (26) over P.
split E CARTSPLIT(PP
D Optimizes (19).
if SPLITTSUCCCEEDDE(() then
SETCHILDREN(P, GETLEFTCHILD(E), GETRIGITCHILD(E))
A GETLEFTCHILD(2) ADDTOQUEUF(Q, GETRIGHTCHILD(E)) end if end while
Output: tree with root node Po end function
POP returns and removes the oldest element of queue a Q. unless Q is empty, in which case it returns NULL. CARTSPLIT runs a multivariate CART split on the pseudo-outcomes P = {pf (pEE,) and either returns a pair of child nodes or indicates that no split of possible.
C.2. In-sample predictions
There is additional bias associated with making predictions based on in-sample observations X; that may have been used either to train the tree structure or to populate the local leaves Lz(x). The recommendation of Tibshirani al. (2024) is along the lines of the out-of-bag mechanism used by Breiman (2001). For an in-sample observation x' {X;) calculate weights a. (20((2)) defined as:
(x') based only those trees whose initial subsample (b) does not contain x'. Then the out-of-bag weight is
1(X; E Lo(x')
0
Obis
(20)
ooo (x')
for
Of coo (x')
1(b & IOU
[T(('')
(burr #200)
The in-sample prediction goob(x)) for x' is made by GRF by solving a version of the locally weighted estimating equation (5) using out of bag weights goab(2)))
(z'), 2.00) (x')) arg min 0,0
af oo (z))oe
which preserve the consistency and asymptotic normality of the GRF estimator at in-sample observations.
C.3. Algorithms and Pseudocode
C.4. Simulation Details
Implementation details. We implement the GRE-FPT algorithm in fork of grf (Tibshirani et al., 2024) available at ltmmmliiiarmmccussses sforestt provide an easy to use interface for VCM and HTE estimation, respectively, and we allow the choice GRF-FPT1, GRF-FPT2, or GRF-grad to be controlled via the met hod argument. Code and data for reproducing all experiments and figures are available at https: /tithu.....// cwm/ wwwm///lei////oooooooooooeoeseee wttps//grrreppepeeeeeeeeeeeeeeee eexerrtpeeeeee ionperiment iimets....
coom/dfleis/ooe
The functions grf: im.forest and
Data-generating settings. The different setting for the target effects (x) include a sparse linear setting, a sparse logistic setting with interaction, dense logistic setting. and a random function generator setting. Tables 2 and provide the details of each regime for VCM and HTE experiments, respectively, for the data-generating model (28). These tables also summarize
28

---

Generalized Random Forests using Fixed-Point Trees
Algorithm 2 Stage IRRF-FPT: Training generalized random forest using fixed-point trees
function ATW.GEEEERRAALEE..ADOOEOEEEEERE
Input: samples S, number of trees B
for I..... do set of samples I SUBSAMPLE(S)
of samples JOUIDD JPOPULATE < HONESTSPLIT(Z) tree To
> See honesty: Appendix C.I.
> See Algorithm 1.
leaves La + POPULATELEAVESSTT JPOPULATE)
> See honesty: Appendix C.I.
end for
Output: forest J + (fI end function
POPSLATELEAVES creates collection of subsets (leaves) of the TEOPULATE samples based on the partition rules of tree Th For weight functions (x)))), see WEGHEIGHT in Algorithm 3. For Stage II, see ESTIMATE in Algorithm 3, where estimates (z) are made given a forest F.
Algorithm 3 GRF-FPT: Estimates of 0" (x)
function ESTIMATE
Input: forest F, test observation x E X'
weights a e GET WFIGHTSHHS(FF
Output: (z)), the solution to the weighted estimating equation (5) using weights a end function
function GETWEIGHTS
Input: forest F, test observation æ
vector of weights a +- ZEROS(n)
> Initialize weights; n = =|S] used to train F.
for indices : X; E Ln(x)
afi] += [o))
end for Output: local weights a/|Fl
Weights (4).
end function
Stage of the GRF-FPT algorithm. The procedure ESTIMATE returns an estimate of 0* (xx) given forest trained under Stage I and test observation x; Algorithm 2.
the different settings used to generate the K-dimensional regressors W, = (W.,,... (.,....,,...k) ) For VCM experiments, Wi, ~ (0, 1) for all = I,..., ... K. For HTE experiments, W, X; = x ~ Multinomial (1, 1, (m Ty (z) denotes the underlying probability the sample is observed having treatment level k (1....,K}.
where
Random function generator. The effect functions (x) = RFG(z) under VCM Setting 4 (in Table 2) and HTEE Setting 5 (in Table 3) follow the random function generator design of Friedman (2001). The idea is to measure the performance of the estimator under a variety of randomly generated targets. Each (-) is randomly generated as a linear combination of functions {ge (-)) of the form
20
(1(x) = Σ (z))
where the coefficients {2 are randomly generated from a uniform distribution a; ~ u([-1,1) Each (((2)) is function of a randomly selected pi-size subset of the p-dimensional variable x, where the size of each subset pa randomly chosen by Pe min([1.5 re] p), and ry is generated from exponential distribution with mean 2, rzz Exp(0.5). Each g(2e) uses pi-sized random subset 24 Ip of the p-dimensional input E R2:
2t = (.., E RR
29

---

Generalized Random Forests using Tixed-Point Trees
Parameter
Values
Parameter
Values
K
4; 16; 64; 256
K
4; 16;
10,000: 20,000; 100,000
1000; 4000
dim(2)
5
dim(2) nTrees
2
Trrees
100
100; 500
Table 1: Parameter values for VCM and HTE experiments in Section 6. Target/regressor dimension K, number of observations
dimension the auxiliary variables dim(X), and number trees nrees.. Experiments include large-n setting (left table) and small-r setting (right table).
VCM Setting
Effect function (z)
Wi,
(+(x BRIJI, BEI (0,
(0 1)
2 (x(x) 31
PRI, Pk2 ~ (0,
((0
3
(ī(x) s(3, x), for 3x ~ (0,1)
(x) = RFG(z)
N(0,
Table 2: Settings for the true effects (-). and the regressors Wi, for VCM experiments in Section 6. The function s(u) = 1+ (1
e 1/3:)- is logistic-type function in (Athey al., 2019). The random function generator RFG(z) is described in Appendix C.4.
HTE Setting Treatment effect (z) (x) = BKIT1, AKI ~ (0,
Treatment probability (x( for Wi.
<x(x) = 1/K for all k.
(1( BRI3I, BRI 100,
*x(x)
21
k = 1, K-1 (1 (1-
1
= .,...,K K
(ī(x) = (((k22t)) for 3kL, 3x2 ~ NO,0
(x(x) = 1/ K for all k.
(x) = 53 x),, for Bx ~ (0, m)(x) =
21
1,
h: 2, ..... K. x1
K-
exp 2i m
(x) = (F)
Σ exp ( for 7k N2
Table 3: Settings for the underlying treatment effects (-) and treatment probabilities (xx for HTE experiments in Section 6. The function (u) == 1+ 140(--2(011/)))
logistic-type function used in (Athey et al., 2019). random function generator RFG(x)
described in Appendix C.4.
such that {de(1)..... } is length-p: permutation of indices drawn from {1...., .p]. without replacement. The functions ge(-) are Gaussian functions of the pe sampled variables:
ge(2t) exp {
{-2 [2(e mD) Ve(z me)
where the mean vector μe E R is randomly generated from standard multivariate Gaussian, Prr (0, The X pr covariance matrix Vi are formed through the spectral decomposition:
V. = U.D.UJ.
where Up a random pe pe orthonormal matrix and D, diag(d1,e..... dpe.t) with diagonal entries dz.e generated from uniform distribution according to /dze 1.
D. Additional Simulations
D.1. Settings for the criterion value experiment in Section 3.4
The criterion value experiment in Section 3.4 was run under a varying coefficient model of the form
Y = W. 0" (X,)) 6G G a~N(0,
(64)
30

---

Generalized Random Forests using Fixed-Point Trees
where the regressors W; were generated bivariate standard Gaussian samples W; (2 and the auxiliary covariates generated as standard uniform samples X; 2/(0, 1). The data-gen sattaggnerating coefficient functions were 0* (x(x) == (sin(2mz), x) and the criterion values were computed based 1000 samples following (64).
D.2. Supporting experiments for Section 6
Multicollinearity in auxiliary covariates. We conducted a VCM experiment with highly correlated auxiliary covariate features. We ran modified version of VCM Setting by generating auxiliary covariates X, (O, )), where E = ا for w E {0, 0.9). Table 4 provides clear summary of the computational performance of GRF-FPT relative to GRE-grad and statistical accuracy (MSE - multiplied by 100 for readability). All experiments were run over a forest of trees and MSE estimates were computed over 50 replications of the model and evaluated on a separate set of n = 5, 000 samples, carried out using GRF-FPT2 and GRE-grad. These results demonstrate clearly that GRE-FPT remains robust, stable, and computationally efficient, even under high multicollinearrty in X,.
(im(X) n
w
Speedup
100 x MSE grad 100 x MSE FPT2
10,000
64
0.00
2.55
16.60
16.83
10,000
0.50
2.53
15.48
15.47
10,000
64
0.90
2.35
10.95
11.09
Table 4: Effect of multicollineariy in the auxiliary covariates X; on the relative computational gain of GRF-FPT2, as well as the statistical accuracy of both GRF-EPT and GRF-grad estimators.
Subsampling ratio. We carried out experiment to show that the subsample proportion does not affect the com- putational advantage or statistical accuracy of GRF-FPT relative to GRF-grad. We varied the subsampling ratio s/n E {0.25, 0.50, 0.75) under VCM Setting 3 over forest of 10 trees carried out using GRF-FPT2 and GRF-grad. Table 5 summarizes our results, averaged over 50 replications of the model, with a test set of 5,000 samples. These results show clearly that the statistical accuracy (MSE) of GRF-FPI2 relative to GRF-grad does not depend strongly on the subsample ratio.
(2) n
s/n.
Speedup
100 x MSE grad
100 x MSE FPT2
10,000
64
0.25
2.77
2.86
2.90
10,000
64
0.50
3.10
2.91
2.90
10,000
0.75
2.98
3.21
3.19
Table 5: Effect of the subsampling ratio s/n on the relative computational gain of GRF-FPT2, well the statistical accuracy of both GRE-PPT and GRF-grad estimators.
Large sample size. We ran additional experiments clearly show how method scales for very large datasets. Using forest of 10 trees, we tested our method on VCM Setting 3 with sample sizes up to n 500, 000, carried out using GRF-FPT2 and GRF-grad. The results are summarized in Table and demonstrate that, even as the dataset grows very large, our method consistently remains faster than GRF-grad. While the relative speedup slightly decreases at first, it stabilizes towards a consistent advantage as grows n grows sufficiently large, suggesting that the advantage is not bottlenecked by n and maintains a robust advantage at scale.
dim X))
K
n
Speedup
2
256
10,000
4.54
256
20,000
3.59
256
50,000
3.49
256
100,000
3.11
256
200,000
3.04
2
256
500,000
3.08
Table 6: Effect of increasing sample sizes n on the relative computational gain of GRF-FPT2.
31

---

Generalized Random Forests using Fixed-Point Trees
Median fit time: VCM (forests) VCM Setting
Median fit time:: VCM (forests)
VCM Setting 2 K ma
K =18
=32
K=
K
K3
32
230
200
GOOCC GADUE
190
Fittime (seconis)
001 2000
600
400
00002
23
28
2000
2000
00000
2000
1033
1000
grad
gradd FPT1 FPT2
grad FPT1 FP72
grad
grad grad FPT1
grad FPT2
graw FPT1
Method
Method
Median times: VCM (forests) VCM Selting
Median times: VCM Selting
VCM (forests)
-25
K-664
K-2
0000
20
0000
time (seconds)
Fit (seconds
10
Boue e
400
200
00:00
00)
200
0000C
005
100
400
3000
120
600
1000
50
2000
00000
400
2000
000000
100
PP12
9r PPT1 FP12
gr PPT1PP1
gradd 19
gradd 1911
Method
Method
Figure S: Absolute times for VCM timing experiments under the settings in Table 2 and large-n settings in Table 1.
D.3. Supporting figures for Section 6
D.3.1. VCM EXPERIMENTS
Large VCM simulations. Figure illustrates the absolute fit times for the GRE-FPT algorithms under the four VCM settings for (z) described in Table 2 over the large-n settings in Table 1. Across all settings and all dimensions, GRF-FPT is consistently several factors faster than GRF-grad. The speedup factor is summarized in Figure 3, which illustrates the relative speedup of GRF-FPT, calculated as the ratio of GRE-grad fit times over GRF-FPT fit times. Consistent with the observations Section 5, we find that the speed advantage of GRF-FPT increases as the dimension of the target increases.
Figure shows that this speed advantage comes while performing comparably to GRE-grad terms statistical accuracy. Across all settings for VCMs with K dimensional targets, the MSE estimates from GRF-FPT is highly similar the MSE estimates of GRF-grad, while for K 256 dimensional targets sees variation in MSE estimates across the methods. This effect likely reflects the increased variance associated with high-dimensional estimation. In some cases see GRF-FPT1 slightly outperform both GRF-FPT2 and GRF-grad, in other cases we see GRF-grad slightly outperform both GRF-FPT methods, and in others GRF-FPT2 yields the lowest MSE. One sees that these differences are typically small. The key benefit we emphasize is that GRF-FPT is able to achieve nearly identical statistical accuracy with a substantial improvement computational speed.
Small n VCM simulations. Figures 8 and 7 illustrate the absolute fit times and relative speed advantage. respectively, of GRF-FPT under the VCM design of (x) over the small-n settings. One sees that even when more modest, GRF-FPT GRF-grad, with possible outliers under VCM Setting 2 at K 4. We believe this negative relative advantage to be caused by random fluctuations in computation and are not representative of the FPT algorithm itself, particularly in light of the fact that the negative effect vanishes when the number of trees increases from 100 to 500. As one would expect based on the large-n results, the relative advantage tends to increase with increasing K, and generally stabilizes with increasing n. Figure 9 shows that the GRF-FPT speed advantage does not come at any material cost in statistical accuracy, with similar performance GRF-grad across all settings.
consistently offers computational advantage
32

---

Generalized Random Forests using Fixed-Point Trees
MSE estimates: Varying coefficient model (VCM)
50 model replications, 5000 test observations
K 256
VCM Setting
VCM Setting
VCM Setting
VCM Setting
0.55
==10000
0.50
6.5
5.00
0.45
4.75
0.40
4.50
5.50
3.9
100 XMSE
0.350
3.25
5.25
3.7
==00000
nTrees -000
3.00
5.00
2.75
0.300
2.50
3.3
0.275
4.50
3.5
3.3
3,4
2.0
=00000
1.9
3.1
1.2
3.0
1.8
grad
FPT1
FPT2
grad
FPT1
FPT2
grad
FPT1
FPTZ
grad
FPT1
FPTZ
Method
MSE estimates:
Varying coefficient model (VCM)
50 model replications, 5000 test observations
K =
VCM Setting 1
VCM Setting 2
VCM Setting
VCM Setting
1.0
1.8
2.00
1.5
1.75
1.2
2
1.50
0.8
0.9
1.0
2.00
100 XMSE
0.9
1.4
1.75
=20000
nTrees -100
1.2
1.50
0.8
0.8
1.25
0.76
1.1
1.2
2.0
1.4
=-00000
0.72
1.0
1.6
0.68
0.9
1.2
0.8
1.0
0.64
0.8
grad
FPT1
FPT2
grad
FPT1
FPT2
grad
FPT1
FPT2
grad
FPT1
FPT2
Method
Figure 6: Estimates of MSE E|](6" (000)6(xxx/A ((X)/K1)) for VCM for K 256 dimensional (top) and = dimensional targets (bottom) under the large-n settings in Table 1.
33

---

Generalized Random Forests using Fixed-Point Trees
Fit time speedup factor: GRF ORF-grad/GRF-FPT (forests)
Varying coefficient model (VCM)
FPT1
FPT2
2.0
1.8
1.5
-=1000
1.2
nTrees = 100
dim(X)
dim(X)
2.0
1.8
n 44000
VCM
Speedup factor
1.6
14
Setting
dim(X)
dim(X) =2
1
2.0
2
1.8
1.6
-=1000
3
1.4
nTrees -
4
dim(X
dim(X)
2.0
1.8
500
1.6
4000
1.4
dim(X]
dim(X)
16
16
R Regressor dimension (K)
Figure 7: Speedup factor for GRE-EPT in comparison GRF-grad for VCM timing experiments under the small-n settings in Table
D.4. HTE experiments
Large n HTE simulations. Figure 11 illustrates the absolute fit times for the GRE-FPT algorithm under the five HTE settings of (z) and TT(()) described in Table over the large-n settings in Table 1. find that GRF-FPT consistently faster than GRF-grad. The speedup factor of GRF-FPT relative to GRF-grad summarized Figure 10, calculated the ratio of GRF-grad fit times over GRF-FPT fit times. As of GRE-FPT scales with the dimensionality K of the target. One sees from both Figures 10 and 11 that GRF-FPT's computational advantage is less dramatic than under the VCM experiments. This can be understood based on the fact that the VCM regressors W₁ are continuous while the HTE regressors represent binary indicators. Continuous regressors provide more granularity when fitting the child statistics fc,. and as result provide a larger set of candidate splits over the covariates. Nevertheless, one sees in Figure 10 that the FPT splitting mechanism is still up to 1.5x faster under the largest regressor setting K 256, with more modest, but persistent savings across all settings.
for VCM experiments, the speed advantage
The statistical benchmarks for HTE experiments shown in Figure 12. Consistent with the VCM experiments, sees that the computational advantage of GRF-FPT does not come at the cost of in terms of its statistical accuracy.
Small HTE simulations. Figures 13 and 14 summarize the relative speed advantage and absolute fit times for the GRF-FPT algorithm under the small-n HTE design. Consistent with the large-n HTE experiments the FPT2 mechanism sees a stable computational advantage across all settings, with an increasing effect in increasing K, while the FPT1 mechanism displays a persistent advantage for K = 16 and comparable computational performance for K 4. The more modest relative advantage for the small-n experiments is itself consistent with the VCM small-n experiments, owing in large part due to the smaller values of K. Figure 15 compares the statistical performance of GRF-FPT to GRF-grad, with no material difference between either GRF-FPTI, GRF-FPT2, or GRF-grad's estimation accuracy.
34

---

Generalized Random Forests using Excnt Trees
Median fit times: VCM (forests)
Median fit times: VCM (forests)
VCM S Setting
VCM Setting 2
K 4
K 16
K 4
K 16
0.15
0.2
n 1000
0.10
0.3
- 1000
nTrees 100
nTrees 100
0.8
1.2
2.0
n =4000
Fit (seconds)
Filuma (seconds)
1.5
44000
0.4
0.5
0.6
1.0
0.5
0.0
0.0
0.0
0.0
0.8
0.8
1000
6.6
n- 1000
64
iTreess =500
nTrees 500
0.2
62
0.0
n = 4000
= 4000
FPT1
FPT2
FPT1 FPT2
FPT1
FPT2
FPT1 FPT2
Method
Method
Median fit times: VCM (forests)
Median fit times: VCM (forests)
VCM 5 Setting
VCM Setting 4
K =4
16
4
K 16
0.15
0.15
0.2
n =
0.10
0.10
11000
ifrees 100
nTrees
0.00
0.00
0.0
0.8
0.0
Fit (seconds)
1.0
=40000
Uime (seconds)
1.0
=4000
100
0.4
0.4
0.5
0.5
0.0
0.0
0.0
0.0
0.0
n n=1000
0.6
-11000
inrres
0.2
nTrees
U
=500
500
n 4000
4
10000
2
grad
grad
grad
FPTZ grad FPT1 FPT2
Method
Method
Figure 8: Absolute times for VCM timing experiments under the settings in Table and small-n settings Table
35

---

Generalized Random Forests using Fixed-Point Trees
MSE estimates: Varying coefficient model (VCM)
50 model replications, 5000 test observations
K = 16
VCM Setting
VCM Setting
VCM Setting
VCM Setting
3.5
3.0
3.0
n 1-00000
14
2.5
mnrresss
2.0
1.1
2.75
2.0
1.2
2.50
2.25
2.00
4000
100 100 XMSE
1.75
1.75
0.9
1.50
3.0
n -1000
2.00
1.2
2.0
1.75
2.0
1.50
1.00
0.95
1.4
C0000 0
0005
1.3
0.90
0.85
1.1
0.80
grad
FPT1
FPT2
1.0
FPT1
FPT2
FPT1
FPT2
1.0
FPT1 FPT2
Method
MSE estimates:
Varying coefficient model (VCM)
50 model replications, 5000 test observations
K
VCM Setting
VCM Setting
VCM Setting
VCM Setting
4.0
3.5
n -1000
2.5
2.5
2.5
2.0
2.0
2.4
3.00
2.75
n -4000
2.75
2.50
2.50
100 XMSE
2.0
2.0
2.00
2.75
3.2
2.8
n n-1000
2.25
2.4
2.00
2.5
1.75
2.0
2.0
2.2
0 1000
19
2.0
18
2.0
1.8
grad
FPT2
grad
FPT2
grad
FPT2
grad
FPT1
FPT2
Method
Figure 9: Estimates of MSE [[[ (X) - ((X)/KI)) for VCM for K 16 dimensional (top) and K 4 dimensional targets (bottom) under the small-n settings Table
36

---

Generalized Random Forests using Fixed-Point Trees
Fit time speedup factor: GRF grad//RF--FFP (forests) Heterogeneous treatment effects (HTE)
FPT1
FPT2
dim(X) 5
dim(X) 5
1.5
1=10000
1.4
1.3
1.2
1.1
1.0
HTE
Setting
Speedup factor
dim(X] 5
dim(X) =
nTrees
200000
1.3
1.2
100
1.1
1.0
1.6
dim(X) 5
dimlX) 5
1.5
1.4
100000
1.1
1.0
64
256
16
Regressor dimension (K)
Figure 10: Specdup factor for GRF-EPT in comparison to GRF-grad for HTE timing experiments under the large-n setting in Table
E. Additional Examples
E.1. Pseudo-outcomes for nonparametric regression
Consider the task of estimating the conditional mean function 6* (x) == E z]. The target 6* (x) is identified by
moment condition of the form (1) with scoring function (Y.) == Y; 0, the residual associated with using as the local
estimate with respect to the i-th sample. The local solution 0p over P is the mean observed response over the parent,
0p=Pp=
Σ Y,
(#X.EP)
The fixed-point pseudosoutcomes are simple the (negative) residuals that result from fitting (6) over P
E -Y; 0p) -Y,
The gradient of the score function is Vo)o(() 11 and hence Ap 11 Therefore, up to a constant factor, the gradient-based pseudo-outcomes p grad for conditional mean estimation reduce to their fixed-point counterparts oerr
grad - Ap A (Y:) = Y; - Y -
In this special case, we recover the conventional splitting algorithm used for univariate responses (Breiman et al., 1984; Breiman, 2001) or for multivariate responses (De'ath 2002; Segal, 1992). Trees grown using !! p or Y, will be identical to one another because CART splits are scale and translation invariant with respect to the response.
More generally, for targets 8* (x) beyond the conditional mean, the form of p will be equivalent p whenever the target function 0* X + θ is map from the input space X' a one-dimensional parameter space θ.
37

---

Generalized Random Forests using Tixed-Point Trees
Median fit time: HTE (forests)
Median fit time:: HTE (forests) HTE Setting 2
HTE Setting 1 <=<
K=1
Km
250
K-
Km004
<
Fittine (seconas)
200
5000
23.3
100
50
0000
5.0
80
001
500
400
2000
0.255
100
392
2500
200
100
1000
500
gred grad
grad FPT1 FPT2
gred FPT2
grad FPT1
ared FPT
grad FPT1
grad FPT1
Method
Method
Median limes: HTE (forests) HTE Selting 3 K-4
Median limes: HTE (forests) HTE Setting --4
255
<=64
10000
(seconds)
Fittime (seconds
150
00) S0H0
-000 mInCc
00002
500
400
1000
2000
1500
00000
700
300
1333
100
500
100
grad PPT1
gr PPT1 PPT2
grad PP12
grad FPT1FFTZ
gred 7PT17P1:
grad PPT1
Method
Method
Median times:
(forests]
Setting 5
K-26
20
00001
Fimmiee (secards)
25
3.0
18
2.5
O0L
6588
100
200
100
Method
Figure 11: Absolute fit times for HTE timing experiments under the settings in Table 3 and large-n settings in Table 1.
F. Real Data Comparison: California Housing
Data. The California housing data appeared Kelley Pace Barry (1997) and can be directly obtained from the Carnegie Mellon StatLib repository (https: //ib. stat cmu. edu / datasett s/ houses. zip). The data includes 20640 observations, where each observation corresponds measurements over an individual census block group in California taken from the 1990 census. A census block is the smallest geographical area for which the U.S. Census Bureau publishes sample data, typically with population between 600 and 3000 people block. Each observation from the California housing data set contains measurements of variables: median housing value (dollars), longitude, latitude, median housing age (years), total rooms (count, aggregated over the census block), total bedrooms (count, aggregated over the census block), population (count), houscholds (count), median income (dollars).
Model.
We consider a varying coefficient model of form
Y; V" vX,, + O + 0g(
(65)
where we suppose that our effects are local to spatial coordinates x = (latitude;, longittuedeee Y; denotes the log median housing value of the census block, and the primary regressors W, = (W. W%,6) passed to the model were median housing age, log(total rooms), log(total bedrooms), log(population). log(households). and log(median income). Here, each g (z) denotes the gcographically-vary varying effect of the corresponding regressor W,,. k. for k: = 1,..., 6. The empirical distribution the transformed regressors passed to each of the GRF models is seen in Figure 17.
38

---

Generalized Random Forests using Excet-Point Trees
MSE estimates: Heterogeneous treatment effects (HTE) 50 model replications, 5000 test observations
K 256
HTE Setting 1
HTE Setting 2
HTE Setting
HTE Setting 4
HTE Setting 5
20
20.0
17.5
18
16
30
1=00000
15.0
25
12.5
20
10.0
16
13
100 XMSE
mirees
15
12
20.0
000004
17.5
00000
-100
15.0
12
3.0
14
5.0
8.5
13
100000
12
1000
7.5
11
grad FPT1 FPT2
grad FPT1 FPTZ
grad FPT1 FPT2 Method
grad FPT1 FPT2
grad FPT1 FPT2
MSE estimates: Heterogeneous treatment effects (HTE)
50 model replications, 5000 test observations
HTE Setting 1
HTE Setting 2
HTE S Setting
HTE Setting 4
HTE Setting 5
1.75
3.0
1.50
n=10000
2.0
1.25
1.5
1.00
1.0
2.5
2.0
100 XMSE
n 10000
2.0
1.5
1.0
110
1.0
0.5
0.85
1.5
1.2
3.5
0.80
1.3
3.0
0.75
1.1
1.0
100000
0.70
0.8
0.9
1000
1.5
0.60
0.7
grad FPT1 FPT2
grad FPT1 FPT2
grad FPT1 FPT2
grad FPT1 FPT2
grad FPT1 FPT2
Method
Figure Estimates of MSE [[ (X) ((X)/K()) under the large-n settings in Table 1.
256 dimensional (top) and dimensional targets (bottom)
Algorithms. We target GRF estimates ((x) (.) 0o(x)) of
based the GRF-
FPT1 and GRF-FPT P22 algorithms described in Section 6, and compare those to GRF-grad. All forests were fit using the grf:: : forst function, which trains the Stage forest and optionally solves for the Stage Il estimates 0(x) for varying coefficient models (65). All versions fit a forest of 2000 trees, the default settings of the original implementation (Tibshirani al., 2024), a subsample ratio of 0.5, and a target minimum node size of 5 observations.
Algorithm
Training time (sec.)
Speedup factor
GRF-grad GRF-FPTI
19.1
15.4
1.24
GRF-FPT2
12.6
1.52
Table 7: Fit times to train forest of 2000 trees on the California housing data.
39

---

Generalized Random Forests using Fixed-Point Trees
Fit time speedup factor: GRF GRF MBFF-grddGRF-FPP (forests) Heterogeneous treatment effects (HTE)
FPT1
FPT2
1.4
n ==100
1.2
nTrees 100
1.0
dim/X)
dim(X)
1.4
HTE
1.3
n =4000
Setting
Speedup factor
1.2
1.1
1.0
0.9
dimiX) =2
dimiX) =2
1.4
1.3
==1000
1.2
1.1
nTrees 5000
1.0
0.9
dimiX) =2
dimiX) =2
1.4
1.3
= 4000
1.1
1.0
0.9
dimm(X) 2
dimiX) 2
Regressor dimension (K)
Figure 13: Speedup factor for GRF-PP!! in comparison to GRF-grad for HTE timing experiments under the small-n settings in Table
Results. Table summarizes the computational benefit of GRF-FPT applied the California housing data. Figures illustrates the local estimates (xx) made by GRF-FPT2, while Figure 16 illustrates the fits under GRF-FET1 and GRF. grad.
40

---

Generalized Random Forests using Exxdd-Piint Trees
Median fit times: HTE (forests) HTE Setting
Median fit times: HTE (forests)
HTE Setting
=4
=
K
=
2.000
3,075
3,300
1000
542
3.055
1000
00L S00ALL
0.02
3.000
0.00
0.00
0.4
0.8
0.6
Fittime (seconds)
0000 1000-11
Fittiime (seconds)
11=4000
0.2
044
62
02
84
02
0.0
03
0000
02
005 S00ALL
8.0
2.0
1.5
9.8
000
0000
8000
1.0
0.
0.9
grad PPT1 FP12
grad FPT1 FPT2
grad FPT1 FP12
grad FPT1 FPT2
Method
Method
Median fit times: HTE (forests)
Median fit times: HTE (forests) HTE Setting
HTE Setting
=1
K
=
3,000
3.075
0001 UU
0.08
3.050
11000000
00L 500010
0.02
Infees
3.000
Fittime (seconds)
0.3
6.4
2000 11
Fittimme (seconds)
M
0.6
0.4
n-:0000
04
62
0.1
02
02
0.3
0A
000T=1
w.
0.3
02
005 5000114
01
Lo0d u
500
15
1.5
-
1.8
0000
0.
0.9
gred FPT1 FPT2
grad FPT1 FPT2
gred FPT1 FPT2
grad FPT1 FPT2
Method
Method
Median fit times: HTE (forests) HTE Setting K
16
0.04
5.04
1000
8.02
nTrees
0.3
time (seconds)
4000
8.2
0.2
1000
0.1
Iirees =500
1.5
1.0
05
grad PPT1 FP12
grad PPY1 FPT2
Method
Figure 14: Absolute fit times for HTE timing experiments under the settings in Table 3 and small-w settings in Table 1.

---

Generalized Random Forests using ExcntPoiint Trees
MSE estimates: Heterogeneous treatment effects (HTE)
50 model replications, 5000 test observations
K = 16
HTE Setting
HTE Setting
HTE Setting
HTE Setting
HTE Setting 5
12.5
25
10.0
20
20
15
7.5
15
15
12
1000
10
10
15
2.5
n 2000
100
100 100 XMSE
25
20
20
n 10000
15
10
mn
10
10
3.5
3.0
2000
1.5
grad FPT1 FPT2
FPT1 FPT2
FPT1 FPT2 Method
grad FPT1 FPT2
grad FPT1 FPT2
MSE estimates: Heterogeneous treatment effects (HTE)
50 model replications, 5000 test observations
K
HTE Setting
HTE Setting 2
HTE Setting
HTE Setting
HTE Setting
12.5
n --1000
5.0
5.0
2.5
3.5
4.0
3.0
n -40000
3.0
100 XMSE
2.0
2.0
10
10.0
7.5
1-1000
7.5
5.0
2.5
3.5
2.5
0000
grad
grad FPT2
grad FPT2 Method
grad FPT2
FPT1
FPT2
Figure 15: Estimates of MSE ||] (X) ((X/KK))) for HTE for K 16 dimensional (top) and K 4 dimensional targets (bottom) under the small-n settings Table
42

---

Generalized Random Forests using Fixed-Point Trees
California housing data: GRF GRF-FPTI -FPT1
Spatally-varyynn effects on log median house values log Households
Median Housing Age
log Median Income
Socromints
Socmmonts
Socmmonts
Sunt Prarencc
Sunt PINTENEO
Sant P11T
Effact lop valu
San Dicgo
San Dicgo
San Diogo
log Population
log Census Block B Bedrooms
log Census Block Rooms
Eat Farcects
San Farcecs
San Farcsco
Lou Angeles
Lou Angeles
Angeles
San Dieggn
San Degg
San Daego
California housing data: GRF -grad Spatially-vrrying effects on log median house values log Households
Median Housing Age
log Median Income
Socromentz
Socoments
Socmmmnts
Sant Ftatence
Sant EISTENE
Sant FINTENC
Lffect log value
San Degn
Diegon
San Diego
log Population
log Census Block Bedrooms
log Census Block Rooms
Sacamentz
Sacoment
Sacamart
San Flerence
San Farrence
Sant Farteaos
Los Argales
Angales
Los Angakes
San Diego
Sat Deego
Sas Drego
Figure 16: Geographically-varyine local estimates ((x) = (x). (c(x)), fit under GRF-PPTI (top) and GRF-grad (bottom). Results for GPF- are presented in Figure 4 found in Scction 7.
43

---

Generalized Random Forests using Fixed-Point Point Trees
California housing data: Regressor distribution
log Census Block Bedrooms
Median Housing Age
log Census Block Rooms
log Population
log Households
log Median Income
300
Housing Age
200
Corr: -0.320
Com: -0.267
Corr: -0.242
Corr: -0.237
Com:
Median
100
-0.163
10
Census Block Rooms
Cor:: 0.933
Corr: 0.848
Corr: 0.922
Corr:
0.293
log Census Block Bedrooms
Corr: 0.889
Corr: 0.974
Corr:
0.034
Population
Corr:
Corr: 0.053
0.925
8
log Households
Corr: 0.075
log Median Income
10 20 30
40
50
6
10
2
8
8
Figure 17: Empirical distribution of regressors from California housing data passed GRF.
44


Reviews:
[{"review_id": "BzpX1zYEJY", "reviewer": "Reviewer_Gxac", "summary": "This paper introduces a computationally efficient alternative to generalized random forests (GRFs) for estimating heterogeneous effects in high-dimensional settings. The authors propose a gradient-free \"fixed-point\" approximation approach that eliminates the need for Jacobian estimation in the GRF splitting criterion, which is both computationally expensive and potentially unstable in high dimensions. The authors show that their method preserves the theoretical guarantees of GRFs (consistency and asymptotic normality) while significantly improving computational efficiency.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> ...Additional real-world applications in different domains could strengthen the practical relevance of the method.\n\n**A1**: We agree with the reviewer. Space limits our discussion here. In the camera-ready version, we will add real-world applications from different domains. \n\n> ...Some additional experiments designed to test robustness to multicollinearity would be valuable.\n\n**A2**: \nWe conducted an additional experiment using the VCM with highly correlated features. We modified our original VCM Setting 3 by generating correlated features $X_i \\sim N(0,\\Sigma)$, where $\\Sigma_{i,j} = \\omega^{|i-j|}$ for $\\omega = 0, 0.5, 0.9$. \n\nBelow is a clear summary of computational performance (speedup factor of GRF-FPT over GRF-grad) and statistical accuracy (MSE, multiplied by 100 for readability). Results are averages over 50 replications with \n$n=10,000$, tested on a separate set of $5,000$ samples using a forest of 10 trees:\n \n|dim($X$) |$n$ |$K$ | $\\omega$| Speedup| 100xMSE ``grad``| 100xMSE ``FPT``|\n|-:|-:|-:|-:|-:|-:|-:|\n|5  |10000 |64 |  0.00| 2.55| 16.60| 16.83|\n|5  |10000 |64 |  0.50| 2.53| 15.48| 15.47|\n|5  |10000 |64 |  0.90| 2.35| 10.95| 11.09|\n\nThese results demonstrate clearly that GRF-FPT remains robust, stable, and computationally efficient, even under high multicollinearity. We will include this analysis in the revised manuscript.\n\n> The paper could benefit from more discussion of the hyperparameter sensitivity of the approach, particularly regarding the choice of subsample size in the honest splitting procedure.\n\n**A3**: Because our method changes only the splitting criterion for the recursive partitioning, both GRF-FPT and GRF-grad share exactly the same subsample size $s$. We do not expect subsample size to change the relative computational advantage or statistical accuracy of our method.\n\nTo demonstrate, we ran an experiment varying the subsample ratio $s/n$ from 0.25 to 0.75 using  VCM Setting 3. The table below summarizes our results (averaged over 50 replications, with a test set of 5,000 samples):\n\n|dim($X$) |$n$ |$K$ | $s/n$| Speedup| 100xMSE grad| 100xMSE FPT|\n|-:|-:|-:|-:|-:|-:|-:|\n|2  |10000 |64 | 0.25| 2.77| 2.86| 2.90|\n|2  |10000 |64 | 0.50| 3.10| 2.91| 2.90|\n|2  |10000 |64 | 0.75| 2.98| 3.21| 3.19|\n\nThese results show clearly that the statistical accuracy (MSE) of GRF-``FPT`` relative to GRF-grad does not depend strongly on the subsample ratio. \n\n> ...A deeper discussion of the substantive insights gained from the spatially varying coefficient estimates would enhance the applied contribution.\n\n**A4**: Our model (eq 36) clearly shows how the influence of housing features changes across different locations in California. Our results from the housing data uncover interesting geographic patterns.\n\nFig 5 clearly illustrates this point. In major urban centers such as LA, San Francisco, and Sacramento, housing prices decrease as the number of households increases. This likely reflects overcrowding in densely populated areas. In contrast, rural regions show the opposite trend: prices rise slightly as the number of households grows. This suggests that, in sparsely populated rural areas, a modest increase in households makes these places more attractive and livable.\n\nPopulation size, however, consistently shows a negative effect on housing prices across nearly all of California, with less geographic variation. Higher populations generally mean lower housing prices, highlighting broader statewide pressures on housing affordability.\n\n> Have you explored how the computational gains scale with very large datasets? ...\n\n**A5**:  We ran additional experiments to clearly show how our method scales for very large datasets. Using a forest of 10 trees, we tested our method on VCM Setting 3 with $K = 256$, dim($X$) = 2, and data size $n$ up to $500,000$. The results appear below:\n\n|dim($X$) |$K$ |$n$ | Speedup|\n|:-|:-|:-|-:|\n|2 |256  |10000  | 4.54|\n|2 |256  |20000  | 3.59|\n|2 |256  |50000  | 3.49|\n|2 |256  |100000 | 3.11|\n|2 |256  |200000 | 3.04|\n|2 |256  |500000 | 3.08|\n\nThese results clearly demonstrate that, even as the dataset grows very large, our method consistently remains faster than GRF-grad. While the relative speedup slightly decreases at first, it stabilizes at a consistent advantage as $n$ grows very large. Thus, our method has no clear computational bottleneck and maintains a robust advantage at scale.\n\n> ...Are there any practical upper limits to K where memory or other constraints might become problematic?\n\n**A6**: As the dimension $K$ grows large, memory usage increases for both methods, potentially hitting hardware constraints. Yet our method (GRF-FPT) is clearly less memory-intensive compared to GRF-grad.\n\nSpecifically, GRF-grad scales in memory as  $O$(``number.of.nodes *`` $K^2$) for each tree.  In contrast, our GRF-FPT scales as $O$(``number.of.nodes *`` $K$). Thus, GRF-``FPT`` remains practical even when   $K$ becomes large, and memory constraints become significant.", "claims_and_evidence": "The claims about computational efficiency and maintained statistical accuracy are well-supported by both theoretical analysis and extensive empirical experiments. The authors provide formal proof of consistency and asymptotic normality, and the experiments show clear speed advantages across various problem settings while maintaining comparable statistical accuracy.", "methods_and_evaluation_criteria": "The proposed methods are appropriate for the problem at hand. The evaluation using both simulated data and real-world California housing data provides a good test of the method's performance in controlled and practical settings. The comparison metrics (computation time and estimation error) are appropriate.", "theoretical_claims": "I checked the main theoretical claims (Proposition 4.1, Theorem 4.2, and Theorem 4.3) and the proofs appear sound. The authors rigorously show that their fixed-point approximation maintains the same asymptotic properties as the original gradient-based approach.", "experimental_designs_or_analyses": "The experimental design is sound, involving a comprehensive comparison across different settings (varying coefficient models and heterogeneous treatment effects) with different dimensionalities. The analysis of computation time and statistical accuracy is appropriate and thorough.", "relation_to_broader_scientific_literature": "The work builds directly on generalized random forests and is well-contextualized within the literature on heterogeneous effect estimation and adaptive partitioning methods. The authors clearly explain how their method relates to and improves upon existing work.", "essential_references_not_discussed": "The paper provides a comprehensive review of relevant literature. No major omissions were found.", "other_strengths_and_weaknesses": "Strengths:\n1. The paper addresses a significant practical issue: the computational cost and potential instability of gradient-based splitting criteria in GRFs when dealing with high-dimensional targets. This is particularly important for applications where the treatment space is large, such as multi-arm experiments.\n2. The theoretical contribution is strong, with clear proofs showing that the fixed-point approximation maintains the same asymptotic properties as the original GRF method while significantly reducing computational complexity.\n3. The method is conceptually elegant and simple to implement, requiring only minor modifications to existing GRF algorithms. The authors carefully explain how their approach fits within the two-stage framework of GRFs.\n\n\nWeaknesses:\n1. The real-world application, while illustrative, is somewhat limited in scope (only one dataset). Additional real-world applications in different domains could strengthen the practical relevance of the method.\n2. The paper mentions but does not deeply explore how the approach behaves in the presence of highly correlated features, which is a common challenge in real-world applications. Some additional experiments specifically designed to test robustness to multicollinearity would be valuable.\n3. The paper could benefit from more discussion of the hyperparameter sensitivity of the approach, particularly regarding the choice of subsample size in the honest splitting procedure.\n4. The qualitative interpretation of the results from the California housing dataset is somewhat limited. A deeper discussion of the substantive insights gained from the spatially varying coefficient estimates would enhance the applied contribution.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. Have you explored how the computational gains scale with very large datasets (e.g., millions of observations)? Are there any potential bottlenecks in scaling the approach to such datasets?\n2. The method shows particular advantages when the target dimension K is large. Are there any practical upper limits to K where memory or other constraints might become problematic?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "So1nJDwGqI", "reviewer": "Reviewer_fj8w", "summary": "The paper introduces a computational improvement to generalized random forests (GRFs). In the tree-growing algorithm of GRFs, the paper proposes to replace the gradient-based approximation for evaluating the split criteria by a more computationally efficient fixed-point based approximation. Theoretical guarantees of statistical effectiveness of the proposed fixed-point approximation are established. Simulation experiments show the proposed algorithm can speed-up training without compromising accuracy performance, especially in dimensional settings.\n\n## update after rebuttal\n\nI thank the author(s) for their detailed rebuttal, which solves many of my concerns. Given that I have already given a positive rating, I will keep my score unchanged.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Why is the saving not $O(K^3)$ \n\n**A1**: In the general case, removing the inverse $A_P^{-1}$ should indeed provide a computational saving of order $O(K^3)$. However, for the specific applications discussed in our paper—such as varying coefficient models (VCM) and heterogeneous treatment effect (HTE) models—the product $\\xi^\\top A_P^{-1}\\psi_{\\hat\\theta_P,\\hat\\nu_P}(O_i)$ (eq 20) has a simple analytical form without the need for a full matrix inversion, making the computational saving scale instead as $O(K^2)$, due to the required matrix-vector multiplications. In general settings without an analytical form for $A_P^{-1}$, our method would indeed yield $O(K^3)$ savings.\n\n> In (27) an approximation to $\\hat\\theta_P$, how would this affect the theoretical results in Prop. 4.1?\n\n**A2**: The approximation does not affect our theoretical results. Here we provide a brief and direct argument. Consider pseudo-outcomes defined with the one-step approximation $\\tilde\\theta_P$:\n$$\n\\phi_i^{FPT} := -(W_i - \\bar W_P)(Y_i - \\bar Y_P - (W_i - \\bar W_P)^\\top \\tilde\\theta_P).\n$$\nLet $\\bar\\theta_{C_j}$ be the child node estimator derived from these approximate pseudo-outcomes. A direct calculation shows that the difference between our original estimator $\\tilde\\theta_{C_j}$ and this approximate estimator $\\bar\\theta_{C_j}$\nsatisfies\n$$\n||\\tilde\\theta_{C_j} - \\bar\\theta_{C_j}|| = ||n_{C_j}^{-1}\\sum_{i \\in C_j} (W_i - \\bar W_P)(W_i - \\bar W_P)^\\top (\\hat\\theta_P - \\tilde\\theta_P)|| = ||A_{C_j} (\\hat\\theta_P - \\tilde\\theta_P)||\\leq ||A_{C_j}|| || \\hat\\theta_P - \\tilde\\theta_P||,\n$$\nwhere $A_{C_j} := n_{C_j}^{-1}\\sum_{i \\in C_j} (W_i - \\bar W_P)(W_i - \\bar W_P)^\\top$.\nUnder GRF's conditions, in a limit where $n_{C_j}\\to\\infty$, $A_{C_j}\\stackrel{p}{\\to}Q$ for a PSD matrix $Q$, and hence $||A_{C_j}|| = O_P(1)$. Let $W_P$ and $Y_P$ denote the centered observations with $i$-th rows $W_i - \\bar W_P$ and $Y_i - \\bar Y_P$, respectively. Then\n$$\n||\\hat\\theta_P - \\tilde\\theta_P|| = ||(A_P^{-1} - \\gamma\\mathbb I) n_P^{-1} W_P^\\top Y_P|| \\leq ||A_P^{-1} -\\gamma\\mathbb I|||| n_P^{-1} W_P^\\top Y_P||\n$$\nUnder GRF's conditions, in a limit where $r := \\sup_{i:X_i\\in P}||X_i -\\bar X_P|| \\to 0$ (the radius of the parent node) and $n_{C_j}\\to\\infty$, we have $||n_P^{-1} W_P^\\top Y_P|| = o_P(r, 1/\\sqrt{n_P})$,\nand $ ||A_P^{-1} - \\gamma\\mathbb I|| = O_P(1)$.\nTherefore $||\\hat\\theta_P - \\tilde\\theta_P|| = o_P(r,1/\\sqrt{n_P})$ and so\n$$\n||\\tilde\\theta_{C_j} - \\bar\\theta_{C_j}|| \\leq ||A_{C_j}|| || \\hat\\theta_P - \\tilde\\theta_P|| = o_P(r,1/\\sqrt{n_P})\n$$\nThis means the approximation does not alter the asymptotic behavior of our estimator. The theoretical results in Prop. 4.1 remain fully valid and unaffected.\n\n> How to generalize approximation (27) beyond  linear regression?\n\n**A3**: The closed form approximation (27) specifically applies to linear-like settings--such as VCM and HTE models--where $\\hat\\theta_P$ can be replaced by the one-step exact-line-search approximation $\\tilde\\theta_P$ (27).\n\nIn a more general setting, such approximation is not available, but one can still use the general fix-point approximation, which would still lead to computational gain. \n> additional settings. 1)  Small sample size\n\n**A4**: To address this, we have run an experiment for small n with data generated according to VCM Setting 3, tested on a separate set of 1,000 observations over 50 replications of each model using a forest of 50 trees\n\n|n|K|Speedup|100xMSE grad|100xMSE FPT|\n|:--|:--|--:|--:|--:|\n|1000|16|1.82|2.97|2.94|\n|2000|16|1.75|2.33|2.32|\n|4000|16|1.74|2.00|1.99|\n\nIn all cases, both the exact and approximate FPT methods are still observe a computational gain relative to GRF-grad, with no material impact on MSE\n\n> additional settings. 2) Correlated W\n\n**A5**: The case of correlated W has been already considered for HTE models in App C.4, Table 2. \n\n> additional settings. 3) Nonlinear models beyond VCM and HTE\n\n**A6**: Due to space constraints, we will demonstrate other nonlinear models in the final manuscript.\n\n> Fig 4 and 18, it seems that the speed gain for VCM is higher than HTE. Explain?\n\n**A7**: The computational gain is higher for VCM than HTE because VCM uses continuous regressors $W_i$, while HTE uses categorical ones. Continuous regressors provide a larger set of candidate splits. Evaluating these extra splits makes the gradient-based method more computationally demanding, so the simpler FPT approach achieves greater relative savings. \n\n> comment on the negative computational gain\n\n**A8**: It appears to be caused by random fluctuations in computation rather than by the FPT algorithm itself. To clarify this, we repeated the experiment five times under the same conditions. The median computational gain is clearly positive, as shown below\n\n|Model|Setting|Trees|dim(X)|n|K|Speedup|\n|:--|:--|:--|:--|:--|:--|--:|\n|HTE|5|50|5|10000|256|1.38|\n\nThese stabilized results are consistent with the gains observed in other HTE experiments", "claims_and_evidence": "* Claims in the paper are in general sound.\n* Simulation studies miss some aspects, making the results less convincing. Please see my comments in “Experimental Designs Or Analyses”.\n* Some additional clarification on the methodology would be appreciated. Please see my comments in “Methods And Evaluation Criteria”.\n* In Section 8, the paper claims that the computational saving scales as $O(K^2)$. Why is the saving not $O(K^3)$ as the computational cost of $A_P^{-1}$ is eliminated?", "methods_and_evaluation_criteria": "While the proposed algorithmic modification is technically sound, I do have two questions on the methodology:\n* In (27) the paper proposes an approximation to $\\hat{\\theta}_P$, how would this affect the theoretical results in Proposition 4.1?\n* How would the approximation (27) be generalized to beyond the settings of linear regression?", "theoretical_claims": "The theoretical results are stated in a rigorous manner. The main assumptions and proofs for Theorems 4.2 and 4.3 are directly taken from Athey et al. (2019), and do not seem to have issues. I briefly reviewed the proof of Proposition 4.1, and it looks reasonable to me, though I did not carefully examine it line by line.", "experimental_designs_or_analyses": "The simulation studies are overall well-deigned, except that some aspects are not discussed or investigated:\n* It would be interesting to see the model performance in terms of speed and accuracy in additional settings. For example,\n\t* Small and moderate sample size\n\t* Correlated $W$: It would be interesting to see how the findings in Figure 1 are translated to actual performance difference.\n\t* Nonlinear models beyond VCM and HTE.\n* Comparing Figures 4 and 18, it seems that the computational gain for VCM is higher than HTE. Could you provide any insights on this?\n* Could you comment on the negative computational gain in Figure 18 under the setting of RFG function, $K = 256$, $n = 10000$, and Trees = 50?", "relation_to_broader_scientific_literature": "The paper presents an interesting computational improvement to the tree-growing algorithm used in GRF (Athey et al., 2019). The modification can effectively speed up the model fitting procedure by removing the need to inverse Jacobian matrices, while preserving the theoretical properties of the original GRF models. Hence, it provides a novel and solid contribution to the GRF literature in my opinion.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Please see my comments in \"Relation To Broader Scientific Literature\" for assessment on the paper's novelty and contribution.", "other_comments_or_suggestions": "The paper is very well written, except for some minor issues:\n* Line 5 of the abstract: \"which __in__ large dimensions...\"\n* Figures 2-4 provides somewhat duplicated evidence on computational saving. The author(s) may consider consolidating them.\n* In the last paragraph of Section 6: \"Figure 10 in Appendix 10\"", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZrQMfrKQOw", "reviewer": "Reviewer_aR56", "summary": "This paper proposes a method to enhance Generalized Random Forests (GRF) by introducing Fixed-Point Trees (FPT) as a gradient-free alternative for recursive partitioning. The authors highlight the limitations of gradient-based methods in GRF, particularly in handling highly correlated regressors and ill-conditioned local Jacobian matrices. To address these issues, they develop a fixed-point approximation approach that eliminates the need for computing Jacobians and improves the stability of tree splits. Experimental results demonstrate that FPT-based splits lead to more robust partitions compared with gradient-based approaches, especially in scenarios with high regressor correlation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> The proofs from theoretical analysis are majorly motivated by Athey et al. (2019). What are the differences and the challenges?\n \n**A1**:  The main difference is our new splitting criterion based on a fixed-point approximation instead of a gradient-based approximation. Proposition 4.1 in our paper, which proves the asymptotic equivalence of our criterion, differs clearly from Proposition 2 in Athey et al. (2019). Our proof directly addresses the unique challenges posed by removing the Jacobian term. These challenges include establishing asymptotic equivalence and ensuring statistical accuracy without relying explicitly on the gradient information.\n\n> ... lacks a rigorous convergence analysis for fixed-point iteration... does not discuss whether the proposed approach satisfies ... the contraction mapping principle, which are essential for ensuring convergence.\n\n**A2**: We would like to clarify a misunderstanding regarding the role of fixed-point method in our approach.\n\n* Numerical convergence: Our method does not perform iterative fixed-point optimization. Instead, we take inspiration from fixed-point iteration but use only a single step of it to simplify the tree-splitting criterion. This single step gives us simplified pseudo-outcomes. Then, these pseudo-outcomes go directly into a standard CART algorithm, which decides how to split the data.\nBecause we rely on CART, our method inherits CART’s proven numerical stability. The stopping rules for CART are clear and well-established (such as minimum node size: fewer than 5 observations or less than 5% of the parent samples). Thus, numerical convergence does not depend on the contraction mapping and is always guaranteed.\n\n* Statistical convergence: From a statistical viewpoint, Prop 4.1 shows clearly that our simplified fixed-point criterion $\\tilde\\Delta^{FPT}$ is asymptotically equivalent to the original gradient-based criterion. Hence, all statistical convergence guarantees from the original GRF method still hold.\n\n> In Section 3.4, the choice of step size $\\eta$ is arbitrary, with the paper setting $\\eta = 1$ without theoretical justification. This assumption may not always hold and could potentially impact the stability of the splitting criterion.\n\n**A3**: Please refer to our response to Reviewer 4 (DWaw) (A1 and A2) for the justifications supporting the choice of $\\eta = 1$, as well as an explanation of why this choice does not impact the stability of the splitting criterion.\n\n> The paper primarily compares FPT with gradient-based approaches but does not benchmark it against other potential improvements in recursive partitioning...\n\n**A4**: Thank you for raising this point. We would like to clarify a misunderstanding. Our method (FPT) only simplifies the criterion used for tree-splitting in generalized random forests. It does not attempt to improve the recursive partitioning algorithm itself. Both our approach (GRF-FPT) and the gradient-based method (GRF-grad) use exactly the same standard CART algorithm for recursive partitioning.\n\nMoreover, to our knowledge, GRF-FPT and GRF-grad are the only existing methods for approximating the splitting criterion in generalized random forests. Thus, no other comparable methods exist for benchmarking.\n\n> It seems that the contributions are somewhat limited.\n\n**A5**: We respectfully disagree. Our method applies broadly to a wide range of important statistical models—local linear regression (Friedberg et al., 2020), survival analysis, missing data problems (Yifan et al., 2023), nonparametric quantile regression, heterogeneous treatment effect estimation, and nonlinear instrumental variables regression (Athey et al., 2019).\n\nMore generally, our approach works for any model defined by nonparametric estimating equations of the form in eq (1). It is especially useful in high-dimensional settings, where the computational cost of GRF becomes a bottleneck. Our fixed-point method offers a practical and scalable alternative while preserving statistical guarantees. We believe this broad applicability and efficiency represent a clear and substantial contribution.\n\n> ... are there specific scenarios where the gradient-based approach still offers advantages?\n\n**A6**: No. In all our extensive studies, we have not found any clear scenario where GRF-grad outperforms GRF-FPT, either statistically or computationally.\n\n> In Fig 10 of App F, e.g. the MSE of the FPT method is higher than that of the gradient-based method under the conditions of VCM Setting 2. The FPT method is not always better than grad. Please elaborate on these results.\n\n**A7**: Our simulations show that sometimes GRF-grad slightly outperforms GRF-FPT in accuracy, while at other times GRF-FPT slightly outperforms GRF-grad. The differences are typically small (about 3-5\\%). The key benefit we emphasize is that GRF-FPT achieves nearly identical statistical accuracy with a substantial improvement in computational speed.", "claims_and_evidence": "I did not find any problematic claims.", "methods_and_evaluation_criteria": "The proposed methods make sense. However, it lacks somewhat novelty.", "theoretical_claims": "The proofs from theoretical analysis are majorly motivated by Athey et al. (2019). What are the differences and the challenges?", "experimental_designs_or_analyses": "The experiments are comprehensive, including simulation and real-world data.", "relation_to_broader_scientific_literature": "The authors propose a general random forests method to estimate heterogeneous effects in large dimensions, which is a scalable alternative for localized effect estimation in machine learning and causal inference applications.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper effectively addresses the instability issue in traditional gradient-based splitting criteria when dealing with highly correlated regressors, providing a practical solution that enhances tree stability.\n\n2. The proposed method is extensively validated on two structured prediction models, VCM and HTE, demonstrating its versatility across different applications.\n\n\nWeaknesses:\n\n1. The paper lacks a rigorous convergence analysis for its fixed-point iteration method. Specifically, it does not discuss whether the proposed approach satisfies key conditions, such as the contraction mapping principle, which are essential for ensuring convergence.\n\n2. In Section 3.4, the choice of step size η is arbitrary, with the paper setting η = 1 without theoretical justification. This assumption may not always hold and could potentially impact the stability of the splitting criterion.\n\n3. The paper primarily compares FPT with gradient-based approaches but does not benchmark it against other potential improvements in recursive partitioning. This limits the comprehensiveness of the method’s evaluation.\n\n4. It seems that the contributions are somewhat limited.", "other_comments_or_suggestions": "Please see Other Strengths And Weaknesses.", "questions_for_authors": "1. Given that FPT improves computational efficiency, are there specific scenarios where the gradient-based approach still offers advantages?\n\n2. In Figure 10 of Appendix F for example, the Mean Squared Error (MSE) of the FPT method is higher than that of the gradient-based method under the conditions of VCM Setting 2. The FPT method is not always better than grad. Please elaborate on these results.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oFfqclGUCJ", "reviewer": "Reviewer_DWaw", "summary": "This paper presents an approach to improving the computational efficiency of GRF  by replacing the conventional gradient-based tree-splitting criterion with a fixed-point approximation. The authors argue that their method maintains the theoretical guarantees of GRF while significantly improving scalability in high-dimensional settings. The paper includes extensive theoretical analysis, experimental validation on both synthetic and real-world datasets, and detailed comparisons with existing methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> ... explain why the derivative matrix can be replaced by an arbitrary scalar\n\n**A1**: In order to show that choosing a different scalar $\\eta$ does not\nchange the stability or outcome of the splitting process, let us compare\ntwo types of fixed-point pseudo-outcomes. Consider first $\\rho\\_{i}=-\\eta\\_{0}\\xi^{\\top} n_{C_j}^{-1}\\sum_{i\\in C_j} \\psi_{\\hat{\\theta}\\_{P},\\hat{\\nu}\\_{P}}(O_i)$\nwith an arbitrary scalar $\\eta=\\eta\\_{0}$. Next, consider $\\delta\\_{i}=-\\xi^{\\top} n_{C_j}^{-1}\\sum_{i \\in C_j}\\psi_{\\hat{\\theta}\\_{P},\\hat{\\nu}\\_{P}}(O_i)$,\nwhere we choose $\\eta=1$. Clearly, we have the relationship: $\\rho\\_{i}=\\eta\\_{0}\\delta\\_{i}$.\nUsing these pseudo-outcomes, the fixed-point approximations for the\nchild node parameters $\\hat{\\theta}\\_{C\\_{j}}$ become: $\\tilde{\\theta}\\_{C\\_{j}}:=\\hat{\\theta}\\_{P}-n_{C_j}^{-1}\\sum\\_{i:X\\_{i}\\in C\\_{j}}\\rho\\_{i}$\nfor $\\rho\\_{i}$, and $\\bar{\\theta}\\_{C\\_{j}}:=\\hat{\\theta}\\_{P}-n_{C_j}^{-1}\\sum\\_{i:X\\_{i}\\in C\\_{j}}\\delta\\_{i}$\nfor $\\delta\\_{i}$. Then, the difference between child nodes satisfies\na simple scaling relationship\n\n$$\n||\\tilde{\\theta}\\_{C\\_{1}}-\\tilde{\\theta}\\_{C\\_{2}}||^{2}=\\eta\\_{0}^{2}||\\bar{\\theta}\\_{C\\_{1}}-\\bar{\\theta}\\_{C\\_{2}}||\n$$\nthis is simply because\n$$\n||\\frac{1}{n\\_{C\\_{1}}}\\sum\\_{i:X\\_{i}\\in C\\_{1}}\\rho\\_{i}-\\frac{1}{n\\_{C\\_{2}}}\\sum\\_{i:X\\_{i}\\in C\\_{2}}\\rho\\_{i}||^{2}=\\eta\\_{0}^{2}||\\frac{1}{n\\_{C\\_{1}}}\\sum\\_{i:X\\_{i}\\in C\\_{1}}\\delta\\_{i}-\\frac{1}{n\\_{C\\_{2}}}\\sum\\_{i:X\\_{i}\\in C\\_{2}}\\delta\\_{i}||^{2}\n$$\nBased on eq (17), the CART splitting criterion is $\\tilde{\\Delta}(C\\_{1},C\\_{2})=\\frac{n\\_{C\\_{1}}n\\_{C\\_{2}}}{n\\_{P}^{2}}||\\tilde{\\theta}\\_{C\\_{1}}-\\tilde{\\theta}\\_{C\\_{2}}||^{2}$\nfor $\\eta=\\eta\\_{0}$, $\\bar{\\Delta}(C\\_{1},C\\_{2})=\\frac{n\\_{C\\_{1}}n\\_{C\\_{2}}}{n\\_{P}^{2}}||\\bar{\\theta}\\_{C\\_{1}}-\\bar{\\theta}\\_{C\\_{2}}||^{2}$\nfor $\\eta=1$. Again, these criteria satisfy a clear scaling relationship:\n\n$$\n\\tilde{\\Delta}(C\\_{1},C\\_{2})=\\eta\\_{0}^{2}\\bar{\\Delta}(C\\_{1},C\\_{2})\n$$\n\nThis scaling shows clearly that multiplying the splitting criterion\nby any positive scalar $\\eta\\_{0}^{2}$ does not affect the CART splits.\nSince CART only ranks splits based on the criterion--it selects\nthe partition $(C\\_{1},C\\_{2})$ that maximizes $\\Delta(C\\_{1},C\\_{2})$:\n$$\n\\arg\\max\\_{(C\\_{1},C\\_{2})} \\tilde{\\Delta}(C\\_{1},C\\_{2})=\\arg\\max\\_{(C\\_{1},C\\_{2})} \\eta\\_{0}^{2}\\bar{\\Delta}(C\\_{1},C\\_{2})=\\arg\\max\\_{(C\\_{1},C\\_{2})} \\bar{\\Delta}(C\\_{1},C\\_{2}).\n$$\n\nTherefore, choosing a different scalar $\\eta$ does not change the\nstability or outcome of the splitting process. The absolute scale\nof $\\Delta(C\\_{1},C\\_{2})$ does not matter; only its ranking across\ncandidate splits determines the final partition.\n\n\n> ... Including visualizations ... to contrast the gradient-based and fixed-point splitting .... could enhance comprehension\n\n**A2**: Thank you for this valuable suggestion. Below, we provide a simple,\nclear visualization that illustrates why the fixed-point approximation\nis effective for tree splitting, comparing it directly with the gradient-based\nmethod. \n\nWe simulate data from a simple VCM with 1D covariates\n$X\\_{i}\\sim U([0,1])$ and 2D regressors\n$W\\_{i}\\sim N\\_{2}(0,\\mathbb{I})$. Outcomes are generated from: $Y\\_{i} = W\\_{i}^{\\top}\\theta^{\\star}(X\\_{i})+\\epsilon\\_{i}$\nwhere $\\theta^{\\star}(x):=(\\sin(2\\pi x),x)$, $\\epsilon\\_{i}\\sim N(0,1)$\nwith $n=5000$.\n\nTo simplify, we focus clearly on just a single candidate split at\na threshold $t$. We define candidate partitions $C\\_{1}(t):=\\\\{X\\_{i}:X\\_{i}\\leq t\\\\}$\nand $C\\_{2}(t):=\\\\{X\\_{i}:X\\_{i}>t\\\\}$ and denote the true splitting criterion\nby $\\Delta(t)$. We compare $\\Delta(t)$ with three approximations:\n\n1. Gradient based: $\\tilde{\\Delta}^{grad}(t)$ \n\n2. Fixed point with $\\eta=1$: $\\tilde{\\Delta}\\_{1}^{FPT}(t)$ \n\n3. Fixed point with $\\eta=\\sqrt{2}$: $\\tilde{\\Delta}\\_{2}^{FPT}(t)$ \n\nWe plot these criteria against all possible splits $t$ from 0 to 1.\n\n[https://tinyurl.com/bdkd9tpb](https://tinyurl.com/bdkd9tpb)\n\nThe visualization clearly shows:\n\n* The curves for the true criterion $\\Delta(t)$, gradient-based\n$\\tilde{\\Delta}^{grad}(t)$, and fixed-point with $\\eta=1$, i.e.\n$\\tilde{\\Delta}\\_{1}^{FPT}(t)$, are all very close. \n\n* Even the fixed-point criterion with $\\eta=\\sqrt{2}$, i.e. $\\tilde{\\Delta}\\_{2}^{FPT}(t)$,\nalthough scaled differently, identifies the same optimal split.\n\n* A vertical dashed line shows that the optimal split chosen by four methods are the same:\n$$ \n\\arg\\max\\_{t}\\Delta(t)=\\arg\\max\\_{t}\\tilde{\\Delta}^{grad}(t)=\\arg\\max\\_{t}\\tilde{\\Delta}\\_{1}^{FPT}(t)=\\arg\\max\\_{t}\\tilde{\\Delta}\\_{2}^{FPT}(t)\n$$\n\nSince CART chooses splits by ranking candidate splits $t$, the absolute\nscale of each splitting criterion does not matter -- only their rankings\ndetermine the chosen split. Thus, our fixed-point method is effective and stable.\n\n> ... should demonstrate metrics beyond ... a smaller sample size.\n\n**A3**: Please refer to the comments under A5 for Reviewer 1 (Gxac) regarding the larger-scale experiments.", "claims_and_evidence": "All the claims made in the submission are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "The proposed method successfully constructs a causal forest while incorporating the computation of the derivative matrix. However, a more detailed interpretation and justification are needed to explain why the derivative matrix can be replaced by an arbitrary scalar.", "theoretical_claims": "While the theoretical justification is strong, a more intuitive explanation of why fixed-point iteration is effective for tree-splitting would improve accessibility for a broader audience. Including visualizations or diagrams to contrast the gradient-based and fixed-point splitting mechanisms could enhance comprehension.", "experimental_designs_or_analyses": "The simulation results should demonstrate metrics beyond speed improvement, and highlighting enhanced accuracy with a smaller sample size.", "relation_to_broader_scientific_literature": "NA", "essential_references_not_discussed": "NA", "other_strengths_and_weaknesses": "NA", "other_comments_or_suggestions": "NA", "questions_for_authors": "NA", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces a more efficient implementation of generalized random forests in which the original gradient-based approximation for evaluating the split criteria is replaced by a more computationally efficient fixed-point based approximation.

Generally speaking, the reviewers rated the paper positively. I believe that the authors adequately addressed most of the clarifying questions and believe that the additional empirical results will strengthen the paper considerably. Moving forward, I encourage the authors to include the additional empirical results mentioned in their replies to Gxac and fl8w. I additionally encourage the authors to include *short* discussions summarizing their replies A5 to Gxac; A1, A2, and A7 to fj8W; and A1 and A2 (perhaps including the visualization in a supplement) to DWas.
Decision: Accept (spotlight poster)