A kernel balancing approach that scales to big data
Abstract
In causal inference, weighting is commonly used for covariate adjustment.
Procedurally, weighting can be accomplished either through methods that model the propensity score, or methods that use convex optimization to find the weights that balance the covariates directly.
However, the computational demand of the balancing approach has to date precluded it from including broad classes of functions of the covariates in large datasets.
To address this problem, we outline a scalable approach to balancing that incorporates a kernel representation of a broad class of basis functions.
First, we use the Nyström method to rapidly generate a kernel basis in a reproducing kernel Hilbert space containing a broad class of basis functions of the covariates.
Then, we integrate these basis functions as constraints in a state-ofthe-art implementation of the alternating direction method of multipliers, which rapidly finds the optimal weights that balance the general basis functions in the kernel.
Using this kernel balancing approach, we conduct a national observational study of the relationship between hospital profit status and treatment and outcomes of heart attack care in a large dataset containing 1.27 million patients and over 3,500 hospitals.
After weighting, we observe that for-profit hospitals perform percutaneous coronary intervention at similar rates as other hospitals; however, their patients have slightly worse mortality and higher readmission rates.
Introduction
Weighting is commonly used to remove overt biases in observational studies and has useful properties, such as its avoidance of explicit outcome modeling [26] and hence division between study design and outcome analysis [29], as well as its ability in principle to examine multiple outcomes [22].
Typically, weighting approaches rely on regression modeling of the propensity score for probability of treatment assignment [27].
However, this approach relies on strong modeling assumptions, and can yield unstable estimators [18].
A recent balancing approach addresses both of these problems by forgoing propensity score modeling and instead using convex optimization to directly find the weights of minimum dispersion that approximately balance the covariate distributions [45].
Despite these appealing features, current algorithms for the balancing approach are impractical in the large and ever-growing datasets commonly found in health sciences.
We propose a weighting approach blending an efficient kernel balancing method with a state-ofthe-art convex optimization algorithm.
We draw on kernel balancing approaches from [39], [43], and [16], which posit that the outcome regressions are located within a kernel representation of a flexible function space.
We use the Nyström approximation to efficiently compute the kernel basis in linear time and space.
Then, we add the basis functions as linear constraints in a quadratic program (QP), which is efficiently solved via a recent specialized first-order alternating direction method of multipliers [33].
The resulting weights are the stable balancing weights that approximately balance functions of the covariates from the kernel.
We show that the proposed method is computationally efficient, has excellent estimator accuracy, and is practical in large-scale observational studies.
Estimation framework 2. Setup
Our setting is an observational study with n triplets ( X, A, Y ) iid ∼ P , where X ∈ R d is a set of observed baseline covariates, A is a binary indicator for treatment assignment, and Y ∈ R is a real-valued outcome.
Applying the potential outcomes framework for causal inference [23, 28], we write Y i
=
A i
Y 1 i + (1 − A i )
Y
i , where Y ai is the potential outcome of unit i under treatment A = a ; a = 0 , 1 .
In our observational study of hospital profit status and heart attack care, A = 1 if the patient was admitted to a for-profit hospital, and A = 0 if admitted to another type of hospital.
Our goal is to estimate the average treatment effect on the treated (ATT),
[Equation]
Identification of the ATT relies on some standard assumptions: (i) consistency , Y
=
Y a if A = a , (ii) exchangeability , A ⊥⊥ Y
| X , and (iii) positivity , P ( A = 0 | X = x ) >
a.s.
[ P ] (see, e.g.,
Chapter 12 in Imbens and Rubin 17).
Given these assumptions,
[Equation]
The first term is simply the expectation of the observed outcomes of the treated sample.
However, ψ represents the mean conditional potential outcomes of the treated sample under control, which was not observed, and thus its estimation relies on the aforementioned assumptions as well as adjustment for differences in the covariates X .
More formally, let f X |
A = a ( x ) be the density of X given A = a , and let µ a ( X ) =
E [ Y | X, A
= a ] represent the outcome regression function.
Then, ψ = (cid:82) µ 0 ( x ) f X |
A =1 ( x ) dx .
Our goal is to estimate the mean of Y in a control population ( A = 0 ) that has covariate distributions similar to the treated population ( A = 1 ).
In order to estimate the ATT given the propensity score π
( X ) , weighting leverages the equality
[Equation]
which holds under the assumptions of consistency and exchangeability
Mechanically, weighting removes covariate imbalances by giving more or less emphasis to each unit’s Y i via the function π ( X ) 1 − π ( X ) .
A simple yet general weighting estimator for E ( Y 0 |
A = 1) = ψ is given by
[Equation]
2. Two classes of weighting approaches
In the modeling approach to weighting, we explicitly model the unknown propensity score π , typically with logistic regression, and then construct the weights using the estimated (cid:98) π .
We define a normalized version of the classic Horvitz-Thompson estimator as
[Equation]
Model-based weighting estimator performance depends on how accurately (cid:98) π is estimated.
If a parametric model captures the true π , asymptotic normality is attained with fast √ n rates.
However, this requires strong functional form assumptions.
Nonparametric models are a useful alternative requiring weaker assumptions; however, they do not scale well to large datasets [44].
The more recently introduced balancing approach [5, 15, 46] avoids explicit modeling of π and instead directly solves for the weights w i that meet prespecified balance and dispersion criteria, guaranteeing balance and yielding a more stable estimator.
Zubizarreta [46] recently introduced stable balancing weights (SBW), which are the solution to the QP minimize
[Equation]
subject to
[Equation]
where ¯ w = 1 n c (cid:80) { i | A =0 } w i , B 1 , ..., B M are M real-valued basis functions of the covariates spanning the model space containing the unknown true outcome model, and δ b is the tolerance for the maximum imbalance.
By balancing approximately rather than exactly, the approach is more robust to data with limited covariate overlap.
Importantly, the degree of balance is customizable via modifying δ b based on subject matter expertise or a tuning algorithm
2. Kernel balancing
There are many possible models of the relationship between covariates and outcomes, but the true model is unknown.
Thus, it is desirable to adjust for the largest possible set of basis functions { B 1 , ..., B M } spanning a general model space for µ a [3], such as the Reproducing Kernel Hilbert
Space (RKHS)
For a given Mercer kernel K : X × X → R , there exists a corresponding feature map Φ : X → Q given by Φ( x ) =
[ √ λ 1 φ 1 ( x ) , √ λ 2 φ 2 ( x ) , · · · ] , where { λ j } and { φ j } are the eigenvalues and eigenfunctions of the kernel operator, respectively such that (cid:82) K
( x, y ) φ j ( y ) dy =
λ
j φ j ( x ) .
Here, Q is an expanded, typically infinite-dimensional feature space.
We assume that
[Equation]
so the true µ a lies within the expanded feature space Q .
By the “kernel trick" K ( x, y )
=
Φ( x ) ⊤ Φ( y ) , there is no need to compute Φ( · ) explicitly, since balancing only requires the dot product of the transformed feature vectors Φ j ( · ) s. Further, for any loss function L : R 2 → R , monotone increasing function Ω , and some τ > 0 , by the representer theorem [35] the solution of the regularized problem
[Equation]
has the form
[Equation]
n (cid:88) i = α i K ( X i , x ) , for some α i ∈ R .
Let K ∈ R n × n be a kernel matrix such that K ij =
K ( X i , X j ) .
Then, since µ a ∈ H K , the regularized problem (5) (with any nonzero degree of regularization) will produce (cid:98) µ a ( X i )
=
K i · α , where K i · is the i -th row of K .
Admitting the eigenvalue (spectral) decomposition K = U Λ U ⊤ where Λ is the diagonal matrix with an ordered set of decreasing eigenvalues, [16] showed that the linear estimator (2)’s empirical bias we wish to minimize through kernel balancing is given by
[Equation]
where (cid:98) w ∈ R n c in (2), U c ( U t ) are rows of U corresponding to the control (treated) units, and 1 n t is a vector of ones of length n t .
As a result, rather than balancing all n columns of K , Wong and Chan [39] and [16] proposed balancing the first r ≪ n eigenvectors of K ; i.e., the first r columns of U .
While easier than balancing all n columns, this may still be prohibitive for large n due to the cost of computing the eigenvalue decomposition.
Thus, we need an effective approximation method for K .
Scalable and flexible weighting 3. Streamlining the kernel basis calculation
While the kernel approach is a flexible method of balancing covariates, the eigenvalue decomposition of a dense n × n matrix is typically solved with algorithms requiring O ( n 2 ) -space and have O ( n 3 ) - time complexity, making them impractical for large samples However, this can be surmounted via low-rank matrix approximations like the Nyström method [9, 13, 38].
By using only a subset of columns of a large kernel matrix, the Nyström method can efficiently find its low-rank approximation.
First, we sample a set of column indices I m with m ≪ n for a matrix
K ∈ R n × n .
While the Nyström method’s standard form selects the indices in I m are via uniform random sampling, it is possible to use more sophisticated sampling approaches [e.g., 20].
Next, we form a matrix C ∈ R n × c with C ij =
K
( X i , X j ) , i ∈ { 1 , ..., n } , j ∈
I m , and a matrix W ∈ R m × m with W ij =
K ( X i , X j ) , i, j ∈ I m .
Then, the standard Nyström approximation for K is
[Equation]
where W + is the Moore-Penrose (pseudo) inverse of W .
Rank-restricted Nyström approximation.
Small singular values in W may lead to unstable W + , so usually one chooses l < m with the rank-restricted Nyström approximation [e.g., 13].
Assume W admits the singular value decomposition (SVD) as W =
U W Λ W U ⊤ W .
Let W l = U W,l Λ W,l U ⊤ W,l be the rank- l truncated SVD of W .
Then,
[Equation]
as the rank- l Nyström approximation for K .
Extra dimensionality reduction.
Further dimensionality reduction is useful for improving computational speed with large n .
Drawing from Wang et al.
[36], let R = CU W,l Λ − 1 / 2 W,l which admits the truncated SVD as R =
˜
U R,s ˜Λ R,s ˜ V ⊤ R,s for s < l .
Here, ˜ V R,s ∈ R l × s contains the dominant s right singular vectors of R .
Then, we get [Equation]
⊤ where D = R ˜ V R,s ∈ R n × s [Equation] as our rank- s Nyström approximation for K .
Given that s < l < m ≪ n , the time and space complexities of computing (cid:102) K l are O ( m 3 + nml ) and O ( nm + m 2 ) , respectively.
Crucial for scalability, these complexities are now linear in n .
Thus, (7) approximates the eigenvector-based kernel bases much more efficiently.
While optimal settings for s , l , and m are not certain, we draw from Wang et al.
[36] by setting l = ⌈ m/ 2 ⌉ and s = l and gradually decreasing s until the problem becomes feasible.
3. Fast optimization of balancing weights
We now consider solving for the balancing weights in large samples.
The alternating direction method of multipliers (ADMM) is a popular convex optimization solution method thanks to its computational efficiency [4].
Despite its strengths, it cannot detect infeasibility, has convergence rates that are sensitive to parameter and data settings, and loses speed gains on high-accuracy settings.
Recently, [33] introduced the operator splitting solver for quadratic programs (OSQP), a state-of-theart ADMM-based general QP solver that overcomes these limitations and is a convenient method for finding the stable balancing weights in (4).
In our setting, since the coefficient matrix is symmetric quasi-definite and sparse, the algorithm can more efficiently solve the linear system of equations, and since the linear system’s coefficient matrix is always non-singular regardless of step-size parameter values, the algorithm maintains numerical stability without losing speed (see Appendix B for details).
Finally, since the relevant linear system’s coefficient matrix system is fixed across iterations and not dependent on δ , the latter can be tuned much more efficiently via factorization caching.
Moreover, because the weighting solutions do not vary with small changes in δ , warm starting can provide even more efficiency (see Appendix D.5 for experimental results).
3. Implementation and bounds on bias
Assume we have completed the rank- l Nyström approximation for K as in (7).
For the matrix C let
C t ∈ R n t × m and C c ∈ R n c × m be rows of C for the treated and control units, respectively.
Given these, the stable kernel balancing weights can be found by solving the following QP:
[Equation]
subject to Qw = z, z ∈ C ( δ ) , (9) for Q =
[ 1 n c
I n c
C c U W,l ] ⊤ and a set C ( δ ) =
{ z | l ( δ ) ≤ z ≤ u ( δ ) } where l ( δ )
= (cid:2) 1 0 n c
[Equation]
Here, I r ∈ R r × r is the r × r identity matrix, and 1 r , 0 r ∈ R 1 × r are vectors of ones and zeros of length r , respectively; δ = ( δ 1 , ..., δ l ) is a vector of investigator-specified maximum tolerable imbalances;
C t U W,l ∈ R 1 × l is a row vector of means of each column of C t U W,l ; and diag( σ 1 , ..., σ l ) =
Λ W,l ∈
R l × l .
In (9), x ∈ R n c , z, l, u ∈ R 1+ n c + l , and Q ∈ R (1+ n c + l ) × n c .
In Appendix A, we describe computation of the proposed estimator in detail.
While in ADMM form, (9) corresponds to the balancing weights problem in (4) with the objective function being the approximated kernel basis ( CU W,l ) ij as B j ( X i ) and sample variance of w
[Equation]
we have s <
l balancing constraints in the QP.
In the following proposition, we show the worst-case bound of the bias (6) for the weighting estimator resulting from from the optimal solution to (9) (see Appendix C for the proof).
Proposition 3.1.
If ∥ α ∥ 2 < ∞ , then we have the deterministic bound
[Equation]
where ∥ · ∥ 2 and ∥ · ∥ F are the 2-norm and the Frobenius norm, respectively.
Further, if we use the extra dimensionality reduction, then, with high probability, the following bound Bias (
[Equation]
holds, where K s is a rank- s truncated SVD of K and ∥ · ∥ ∗ is the trace norm.
By this proposition, the upper bound on bias is the sum of error due to residual covariate imbalance after weighting and error due to kernel matrix approximation, with the former controllable via the balancing constraints.
Note that W l and K s are the best low-rank approximations for W + and K , respectively, with the corresponding ranks.
The kernel matrix approximation error generally declines with larger s , l , and m , and the bound of the residual error ∥
K − (cid:102) K
∥ 2 can be found; see, e.g., [13].
In Appendix D, we provide extensive simulation studies showing that the proposed approach has excellent estimation accuracy and also demonstrates superior computational efficiency at sample sizes up to 1 million and under a variety of covariate specifications, even relative to logistic regression.
llllllllllllllll l l l l lll lll lll llllllllllllllllll l l ll l l llllllllllllllllll l l l l l l l l l l l l l l l l 0.
Panel A. Stable Kernel Balancing Approach
TASMD Before Weighting
T AS
M D A ft e r W e
i
gh t
i ng 0 .0 .0 .
.llll l l llllllllll l l l l lll
lll lll llllllllllllllllll l l ll l l lllllllllllllllll l l l l l l l l l l l l l l l l l 0.
Panel B. Random Forest Approach
TASMD Before Weighting
T AS
M D A ft e r W e
i
gh t
i ng 0 .0 .0 .0 .
A national study of heart attack care by hospital profit status
The effect of for-profit orientation on medical care is a central question in health economics.
Medicare pays hospitals more if heart attack patients receive percutaneous coronary intervention (PCI), which may incentivize excessive and potentially unneccessary use.
However, a reasonable alternative hypothesis is that profit motives may improve clinical efficiency and spur technical and human capital investments that lead to better outcomes and potentially reduced need for PCI.
In fact, two studies from the 1990s and early 2000s using small data samples reported opposing results [30, 31]; however, these studies had fewer than 160,000 patients from small numbers of hospitals that were participating in voluntary cardiac reporting systems, which may have altered provider behavior.
In contrast, we studied a larger sample of 1,273,636 Medicare beneficiaries admitted to hospitals for heart attack between October 1, 2010 and November 30, 2019.
There were 206,948 patients admitted to 775 for-profit hospitals, while 1,066,780 patients were admitted to 2,763 control hospitals.
The kernel balancing incorporated 87 adjustment covariates describing the patients and the hospitals in their county of residence, with the latter partially addressing selection bias due to patients choosing where to live based on their medical needs.
After weighting, we considered the samples balanced if all target absolute standardized mean differences (TASMD) [6] were below 0.1.
4. Kernel balance results
Panel A of Figure 1 shows that the weighting algorithm successfully balanced all covariates, with all TASMDs after weighting far below our standard of 0.1, and none above 0.03.
The kernel basis computation required only 32.3 seconds, while the OSQP algorithm solved for the stable kernel balancing weights in just 132 seconds.
Other methods encountered difficulties-
the balancing problem was too large for quadprog , while the kernel ridge regression implementation of the modeling approach could not obtain a solution.
While fast random forests via ranger did find a solution in the modeling approach, it required much more time than kernel balancing, and as can be seen in Panel B of Figure 1, the random forest approach was less successful at balancing even just simple covariate means.
[Table]
4. Heart attack treatment and outcomes
Table 1 shows treatment and outcomes by hospital profit status after weighting.
For-profit hospitals and control hospitals used PCI at almost the same rate, which aligns with [30], but conflicts with Sloan et al.
However, the latter used data from 1994-1995, and guideline-based care has become more prominent since, perhaps leading to converging treatment practices.
30-day mortality was slightly but significantly higher, while 30-day readmission was about 1% higher at for-profit hospitals, a significant difference.
Summary and future work
We described a stable kernel balancing approach to weighting that scales to large datasets without sacrificing estimator accuracy.
We showed that the Nyström approximation makes it computationally feasible to find a set of general functions of the covariates in RKHS, and described how the kernel basis from the Nyström approximation can be efficiently balanced via a modern ADMM-based convex optimization method.
Using the method to study the relationship between hospital profit status and heart attack care, we found for-profit hospitals have similar PCI rates, but worse patient outcomes.
Scalable and stable kernel balancing affords new opportunities to leverage the advantages of the balancing approach to weighting for observational studies using big data. There are several avenues for future work.
First, the algorithm requires a choice of kernel and parameters s , l , and m to form the kernel basis.
While our simulations used heuristics from [36], data-adaptive selection methods could be explored, possibly using balance quality after weighting to adjust the parameters to improve balance. The impact of kernel approximation could be studied by re-examining the large-scale results in [39] and [37] after adding the approximation step, giving conditions for the validity of the bootstrap to conduct inference. Finally, while we used simple random sampling for the Nyström approximation, future work could study how more complex sampling schemes affect the kernel approximation bias of the estimator. APPENDIX A Algorithm
input: Sample Z 1 , ..., Z n , vector δ , integers s < l < m ≪ n , and kernel K ( · , · ) *Step 1: Form kernel bases 1 Sample a set I m of m ≪ n indices in { 1 , ..., n } 2 Form C ∈ R n × m with C ij =
K ( X i , X j ) , i ∈ { 1 , ..., n } , j ∈ I m , and W ∈ R m × m with
W ij =
K
( X i , X j ) , i, j ∈
I m 3 Compute the rank- l truncated SVD W l
= U W,l Λ W,l U ⊤ W,l 4 if dimensionality reduction then
Compute the rank- s truncated SVD of R = CU W,l Λ − 1 / 2 W,l as ˜ U R,s ˜Λ R,s ˜ V ⊤ R,s
Compute D = R ˜ V R,s 7 Compute Q, l ( δ ) , u ( δ ) using (9)
*Step 2: Optimize weights with OSQP 8 Choose parameters ρ > 0 , σ > 0 , and α ∈ (0 , 2) ; initialize w 0 , y 0 , z 0 , and ν 0 ; and set k =9 repeat ( (cid:101) w k +1 ,
[Table]
{+k +1 ) ← solve (cid:20) − (1 + σ ) I n c − Q ⊤ − Q ρ − 
+}
I (cid:21) (cid:20) (cid:101) w k + ν k + (cid:21) = − (cid:20) σw k − 1 /n c 1 n c z k − ρ − 1 y k (cid:21) ;
[Table]
[Table]
+
[Table]
w ← w k , k ← k +16 until termination criterion satisfied ; output: (cid:80)
=
Y
,i , where Y c is an outcome vector for the control units
The above OSQP algorithm for obtaining the SBW is particularly well-suited to large-scale problems.
First, the most computationally expensive step of Algorithm 1 is solving the linear system in Step 10; however, since the matrix (cid:104) − (1+ σ ) I nc − Q ⊤ − Q ρ − 1 I (cid:105) is symmetric quasi-definite and sparse (with sparsity 2 n c −
+( Kd +2)(2 n c +1) n 2 c +( n c + Kd +1)(3 n c
+ Kd +1)
≈ 1 n c )
, a number of efficient algorithms can be applied to this step.
One option is the QDLDL factorization procedure [8], which is the default linear system solver in OSQP, while an alternative option is approximate minimum degree ordering [1].
Second, although the investigator should carefully tune δ in the SBW algorithm to obtain the desired degree of balance, for example using the bootstrap [6, 37], there are methods to execute this step with greater efficiency.
Although by necessity the bootstrap requires a number of iterations, the fact that the coefficient matrix used in Step 3 of Algorithm 1 remains unchanged across iterations and does not depend on δ permits the use of factorization caching; i.e., the factorization needs to only be performed once and the resulting information can be stored for use in subsequent iterations.
Also, small changes in δ tend not to substantially affect SBW solutions, so warm starting can be used for added efficiency.
Together, the techniques of factorization caching and warm starting make OSQP a particularly efficient approach to completing the SBW procedure.
Finally, since the coefficient matrix in Step 10 of Algorithm 1 is always non-singular regardless of the value of σ , some arbitrarily small σ can be chosen without a detrimental effect on computation time.
We remark that there exist alternative ADMM-based methods that could be applied; however, these approaches require reformulation of our QP, which introduces many extra coefficients into the optimization procedure and therefore may not be computationally efficient [33].
C Proof of Proposition 3.
Proof.
i) (cid:102) K l = CU W,l Λ − 1 W,l U ⊤ W,l C ⊤ .
Let w t = 1 n t 1 n t and (cid:101) U Λ − 1 W,l (cid:101)
U ⊤ ≡ CU W,l Λ − 1 W,l U ⊤ W,l C ⊤ =
(cid:102) K l .
Then essentially by the
Cauchy–Schwarz and triangle inequalities, we have
[Equation]
For the second term in the last display
[Equation]
it follows that
[Equation]
where the first inequality follows by the Cauchy–Schwarz inequality and the second by the balancing conditions and the fact that sup α ∥ α ∥ 2 < ∞ .
Also for the first term it follows that
[Equation]
Putting the two pieces together, we have the desired bound
[Equation]
It can be shown that DD ⊤ is the rank- s truncated SVD of (cid:102) K l [36].
Hence
[Equation]
holds with probability at least 0 .
9 [36, Theorem 1].
Then result follows by the same logic as part i.
D Numerical Experiment
We study the computational performance and estimation accuracy of several modeling and balancing weighting methods.
We extend the simulation design by Hainmueller [15] to encompass nonlinear treatment assignment processes and sample sizes up to ten million.
D. Study design
Consider an observational study with continuous and binary covariates generated as follows (cid:34)
[Equation]
Using these six covariates, we generate the outcome and treatment variables as
[Equation]
[Equation]
where ε ∼ N (0 , σ 2 ε ) .
By adding more nonlinearity, (11) is extended from the treatment assignment mechanism in Hainmueller [15].
We use σ 2 ε = 30 in our setup.
We consider two covariate specifications with X cor =
( X 1 , X 2 , X 3 , X 4 , X 5 , X 6 ) corresponding to the setting where all covariates are correctly specified, and X mis =
( X 1 X 3 , X 22 , X 4 , X 5 , X 6 ) corresponding to the setting where some covariates are misspecified.
The latter case is intended to yield a misspecified propensity score in the modeling approach, and misspecified covariate balance in the balancing approach.
For each σ 2 ε , we consider each interaction of treatment assignment mechanism and accuracy of covariate specification.
We consider five sample sizes: n ∈ { 2 · 10 3 , 5 · 10 3 , 10 4 , 10 5 , 10 6 } .
In this design, the ATT is equal to zero by construction.
To separate design from analysis and focus on computing scalability and speed, we based the weighting estimator (2) solely on (cid:98) π ; however, it can be augmented by an outcome model to form a doubly-robust estimator as in [25].
D. Methods considered
Estimation of the ATT by weighting requires explicit modeling of π for (cid:98) ψ mod in the modeling approach, or solving the QP (9) to obtain (cid:98) ψ bal in the balancing approach.
We consider various common methods, including some considered useful for large-scale modeling or optimization.
For practicality, we focus on methods that can be executed using readily available packages in R .
For consistent comparisons of balancing estimators, we fix δ = 0 .
across all methods, although in Section D.5 we evaluate the computational performance of the methods when selecting δ in a data-driven fashion using the algorithm in [6].
We consider nine methods for (cid:98) ψ mod and seven methods for (cid:98) ψ bal totaling 16 methods classified into six groups (see Table 2.
For the modeling approach, Group M1 methods use logistic regression (or GLM) to model the propensity score.
Group M2 contains lasso algorithms where the tuning parameter is selected via 10-fold cross validation.
In Groups M1 and M2, the regression models for π add quadratic terms and all pairwise products of the observed covariates.
Group M3 includes three common non-parametric algorithms: random forests, kernel ridge regression, and generalized additive models using splines [21].
For the balancing approach, Group B1 includes the widely-used open-source QP solvers in R .
Group B2 studies two well-known commercial solvers.
Finally, Group
B3 includes three state-of-the-art ADMM-based solvers aimed at reducing processing time and used with the kernel balancing approach.
The kernel bases are constructed using the Gaussian kernel, which projects into an infinite-dimensional extended feature space Q .
For the Nyström approximation, we used m = 100 and set other parameters via the heuristic described in Section 3.1.
For OSQP, we use the same parameters as Stellato et al.
[Table]
B1.
Open-source solvers quadprog Commonly-used QP solver based on the dual method
[14] qpoases
QP solver based on parametric active-set algorithm [10] B2.
Commercial solvers gurobi Large scale commercial optimizer by Gurobi Optimization Rmosek Large scale commercial optimizer by Mosek ApS B3.
ADMM-based solvers osqp
[Table]
for
[Table]
OSQP
[Table]
POGS) scs
[Table]
conic solver
(SCS)
D. Measures and settings
We assess estimator performance using the root-mean-squared error (RMSE) defined by (cid:110) 1 S (cid:80)
Sj =1 ( (cid:98)
ψ j − ψ ) 2 (cid:111) 1 / 2 averaged across S = 100 simulations, and quantify the computational cost in average central processing unit (CPU) time across simulations, which we report in seconds unless otherwise noted.
Unless otherwise mentioned, each method is implemented in R using each package’s internally available solvers and default settings for ease of performance comparisons across methods.
However, some algorithms may perform better under non-default settings.
All simulations were completed using the Harvard Faculty of Arts and Sciences Research Computing (FASRC) clusters, 4 with available memory fixed at 128GB.
In the results tables, we write ‘F’ when the optimization process failed in more than half the simulations, and ‘M’ when the system ran out of memory in more than half the simulations.
[Table]
D. Empirical results
RMSE.
Table 3 shows results under weak overlap.
The balancing estimators (cid:98) ψ bal substantially outperform the modeling estimators (cid:98) ψ mod at all considered sample sizes regardless of covariate specification; however, gaps between (cid:98) ψ bal and (cid:98) ψ mod are somewhat smaller when the covariates are misspecified.
We also note that the RMSE for kernel balancing is the same as or only slightly different from the commercial solvers.
CPU time.
Because the average CPU time did not appreciably vary across simulation settings, we present only their averages in Table 4.
The GLM and OSQP methods were fastest, taking only seconds to find the optimal weights even in the largest sample size of one million.
OSQP was faster than the commercial solvers MOSEK and Gurobi, by about an order of magnitude compared to the latter.
In contrast, the lasso and non-parametric methods required much more time at that sample size, in some cases hundreds of times more than GLM and OSQP.
Among the ADMM-based solvers,
OSQP was much faster than SCS at the largest sample size.
In our approach, forming the kernel basis requires less computation time than solving the QP, demonstrating the efficiency of the Nyström approximation.
Convergence and Memory.
Several methods had computational difficulties at larger sample sizes, particularly (cid:98) ψ bal approaches.
At samples of 100,000 or more, the two open-source solvers failed to converge, while the system ran out of memory when attempting to execute POGS, perhaps due to its inability to accommodate sparse matrices.
Among (cid:98) ψ mod approaches, the kernel ridge regression ran out of memory at 1 million observations.
D. Time reduction via factorization caching and warm starting
Since the vectors l, u are the only parameters that vary in δ in Algorithm 1, we can speed up repeated
OSQP calls via factorization caching and warm starting.
To illustrate this computational gain, we solve a sequence of OSQP-based SBW problems using 100 values of δ equally spaced between 0 .and 0 .
1 on the same data, and compare its computing time against two popular commercial solvers gurobi and Rmosek (with the default setting of interior-point methods).
We measure the cumulative time for each solver at m = 1 , 20 , 40 , 60 , 80 , 100 problem instances across different values of δ .
We repeat the simulation 100 times and present their average values in Web Figure 2.
[Table]
For gurobi and Rmosek , the cumulative time increases linearly with the number of iterations as would be expected; i.e., m iterations took roughly m times as long as a single execution.
However, m iterations of the OSQP algorithm took much less time than m × (the execution time for a single run); in fact, it is on average 2 .
4 times faster.
This 2 .
4 -fold time improvement demonstrates the benefits of factorization caching and [Table]