Bi-Level Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data
Authors: Shuli Zhang
Hao Zhou
Jiaqi Zheng
Guibin Jiang
Cheng Bing
Wei Lin
Guihai Chen
Venue: NeurIPS 2025 poster ()
Keywords: Causal Inference ; Decision-Focused Learning; Marketing Optimization; Observational and Experimental Data
Primary area: deep_learning

Abstract: Online Internet platforms require sophisticated marketing strategies to optimize user retention and platform revenue — a classical resource allocation problem. Traditional solutions adopt a two-stage pipeline: machine learning (ML) for predicting individual treatment effects to marketing actions, followed by operations research (OR) optimization for decision-making. This paradigm presents two fundamental technical challenges. First, the prediction-decision misalignment: Conventional ML methods focus solely on prediction accuracy without considering downstream optimization objectives, leading to improved predictive metrics that fail to translate to better decisions. Second, the bias-variance dilemma: Observational data suffers from multiple biases (e.g., selection bias, position bias), while experimental data (e.g., randomized controlled trials), though unbiased, is typically scarce and costly --- resulting in high-variance estimates. We propose **Bi**-level **D**ecision-**F**ocused **C**ausal **L**earning (**Bi-DFCL**) that systematically addresses these challenges. First, we develop an unbiased estimator of OR decision quality using experimental data, which guides ML model training through surrogate loss functions that bridge discrete optimization gradients. Second, we establish a bi-level optimization framework that jointly leverages observational and experimental data, solved via implicit differentiation. This novel formulation enables our unbiased OR estimator to correct learning directions from biased observational data, achieving optimal bias-variance tradeoff. Extensive evaluations on public benchmarks, industrial marketing datasets, and large-scale online A/B tests demonstrate the effectiveness of Bi-DFCL, showing statistically significant improvements over state-of-the-art. Currently, Bi-DFCL has been deployed across several marketing scenarios at Meituan, one of the largest online food delivery platforms in the world.

Bi-ee Decision-Focused Causal Learning for Large-Scale Marketing Optimization: Bridging Observational and Experimental Data
Shuli Zhang 1-j Hao 1.2 Jiaqi Zheng
Guibin Jiang?
Bing Cheng² Wei Lin²
Guihai Chen
State Key Laboratory for Novel Software Technology. Nanjing University, Nanjing, China Meetuan, Beijing. China
zhangshul www.aa..ma...eeeeeeeeeeeee. edu.ccn (jzheng, gchen)@nju. edw.:eijuuuuuu..... cn (zhouhao29, jiangguibin, bing.cheng, cheng, 1inwei31)@meitee @eeetuan.cc..
Abstract
Online Internet platforms require sophisticated marketing strategies optimize user retention and platform revenue a classical resource allocation problem. Traditional solutions adopt two-stage pipeline: machine learning (MI.) for pre- dicting individual treatment effects to marketing actions, followed by operations research (OR) optimization for decision-making This paradigm presents two fundamental technical challenges. First, the prediction-decision misalignment: Conventional ML methods focus solely on prediction accuracy without considering downstream optimization objectives, leading to improved predictive metrics that fail translate better decisions. Second, the bias-variance dilemma: Observa- tional data suffers from multiple biases (e.g., selection bias, position bias), while experimental data (e.g.. randomized controlled trials), though unbiased, is typically scarce and costly resulting in high- variivareecce estimates. We propose Bi-level Decision-Focuseee Causal Learning (Bi-DFCL. that systematically addresses these challenges. First, we develop an unbiased estimator of OR decision quality using experimental data, which guides ML model training through surrogate loss func- tions that bridge discrete optimization gradients. Second, we establish bi-level optimization framework that jointly leverages observational and experimental data, solved via implicit differentiation. This novel formulation enables unbiased OR estimator to correct learning directions from biased observational data, achiev- ing optimal varianee tradeoff. Extensive evaluations on public benchmarks, industrial marketing datasets, and large-scale online A/B tests demonstrate the effectiveness of Bi-DFCI, showing statistically significant improvements state-of-the-ard Currently, Bi-DFCL. has been deployed across several marketing scenarios Meituan, one the largest online food delivery platforms the world.
Introduction
Marketing is one of the most effective strategies for enhancing user engagement and platform revenue, and such, variety of marketing campaigns have been widely adopted by online platforms. For instance, coupons on Taobao 45) stimulate user activity, dynamic pricing on Airbnb [4] and discounts on Uber [3] encourage increased usage. However, while these actions generate incremental
Both authors contributed equally this research. i Work was done during an internship Meituan. I Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

revenue, they also consume substantial marketing resources such as budget. Due to these constraints, only a subset of individuals (e.g., shops or products) can receive marketing treatments. Therefore, determining how to allocate marketing resources effectively- given that users respond differently to various promotional offers-is crucial for campaign success. This challenge is typically formulated as resource allocation problems, which has been extensively studied in both academia and industry.
The mainstream solutions to these problems are two-stage methods (TSM) 2 48 39
In the first stage (ML), machine learning models are used to predict individual-level (incremental) responses to different treatments. In the second stage (OR), these predictions are fed into combinatorial optimization algorithms to maximize overall revenue. Hence, existing two-stage methods focus on separately optimizing the prediction and the subsequent resource allocation, treating them as decoupled problems. Despite widespread use, TSM suffer from two fundamental technical challenges:
The prediction-decision misalignment: ML focuses on predictive accuracy, while OR aims for decision quality. However, improved prediction accuracy does not necessarily yield better decisions in many predict-then-oppe then-optimize scenarios [23] 14 32), due to the decoupled design. This misalignment is especially pronounced in marketing for two reasons. First, marketing OR problems are typically non-convex and NP-hard resource allocation tasks, which can amplify or accumulate prediction errors from the MI. stage when passed to OR. Second, marketing involves counterfactual challenges (discussed later), making accurate predictions even harder. As result, two-stage methods often lead to suboptimal decisions in marketing optimization.
The bias- variance dilemma: (OBS) data are abundant and easy to collect, e.g., from user behavior logs or transactions. However, such data are inherently biased due to confounding and lack of randomization, leading to high bias and low variance. In contrast, randomized controlled trials (RCTs) [1O] (6) considered the gold standard for causal inference, as randomization provides experimental data that yield unbiased estimates. Yet, RCT data costly and limited in size, resulting in low hias and high variance, which also increases the risk of overfitting and reduces generalization. While OBS and RCT data are complementary, two-stage methods that rely solely on one type naively combine both fail to achieve an effective bias-variance tradeoff, limiting robust decision-making in marketing.
In marketing optimization and causal inference BU. observational
Recently, Decision- Learning (DFL) 23 29 27 [4] has emerged as promising alternative to traditional TSM by integrating ML and OR objectives within an end-to-end framework, specifically designed address the Prediition-decision misalignment. The core idea is train MI. models using a loss function that directly reflects the quality of the resulting decisions. However, applying general DFL methods to marketing optimization raises unique challenges, including complexity of multi-choice knapsack problem (MCKP), constraint uncertainty, counterfactuals, computational cost of lcrge-scale marketing data [52]. To tackle these domain-speciin issues of marketing optimization, two specialized DFL approaches DHCL [51] and DFCL [52]- have been proposed for marketing scenarios. While DHCL and DFCL have made notable progress in narrowing the gap between prediction and decision objectives (Challenge I of TSM), they do not fully resolve this misalignment, and further improvements are needed. Moreover, these methods may even exacerbate the bias- variance dilemma (Challenge 2 of TSM), as will be discussed in detail in Sec. 2
In this work, we propose Bi-Level Decision-Foouus establish a bi-level optimization framework that leverages RCT data to end-to-end train an auxiliary Bridge Network by minimizing our proposed unbiased OR estimator, which in turn dynamically corrects the training direction on OBS data. By bridging OBS and RCT data, this design enables Target Network better capture unbiased task-specific knowledge and address both the prediction reedicionndeciiionn misalignment and variance dilemma in TSM and DFL. We summarize our main contributions as:
Causal Learning (Bi-DFCL). The key idea is to
Bridging the prediction-deeision gap: We propose an unbiased estimator of decision quality within the DFL paradigm and design two innovative surrogate decision losses leveraging RCT data. Such losses enable exact and efficient gradient computation for discrete optimization and, by operating on the primal problem, directly target the actual budget constraints of real-world marketingg-leanee
to more practical and consistent alignment between prediction and decision.
A Adressing the bias- variance dilemma: We establish a bi-level optimization framework that bridges OBS and RCT data. This architecture enables our unbiased OR estimator to dynamically correct the learning direction from biased OBS data via an auxiliary Bridge Network, achieving

---

optimal bias- variance trade-off. We further develop an implicit differentiation- based algorithm for bi-level optimization, ensuring end-to-end differentiability and scalability for large-scale marketing.
Adattive multi-objective loss balancing: By explicitly assigning prediction and decision losses the lower and upper levels of bi-level optimization, Bi-DFCL automatically and flexibly balances these objectives in data-driven manner, eliminating the need for manual hyperparameter tuning.
Comprehensive offline and online validation: We conduct extensive offline experiments on public benchmarks and industrial marketing datasets, as well as large-scale online A/B tests at Meituan, one of the largest online food delivery platforms in the world. Results show that Bi-DFCL consistently outperforms state-of-the- of-the--rt methods. Notably, Bi-DFCL has already been deployed in several real-world marketing scenarios on this platform, generating significant revenue gains.
2
Related Works
Two-Stage Method (TSM). The mainstream approach to the resource allocation problem in marketing typically adopts a two-stage paradigm [3] 33 48 2 [3), in which the machine learning (ML) and operations research (OR) stages are addressed independently. In the first stage, uplift models are employed to predict the individual treatment effects. In the second stage, the resource allocation task formulated as multi-choice knapsack problem (MCKP), which is NP-hard but can be efficiently solved using Lagrangian duality theory 2 3
39 51. Note that the core idea of these methods is to continuously improve the predictive accuracy of the uplift models in the first stage. Accordingly, prior studies have focused on the design of uplift models, which can be categorized into four main groups: meta-learners [17 24], causal forests
18] and representation learning approaches 16 35 19). However, as discussed in Sec. TSM
49
2 reweighting-bent -based methods 48
suffers from misalignment between prediction and decision objectives and fails to achieve an effective clas-variance tradeoff.. Thus, even with improved predictive accuracy from advanced uplift models, better predictive metrics often do not translate into better or more robust decision quality.
Decision-Focuss Learning (DFL). DFL offers an appealing alternative to the traditional two- stage approach by integrating prediction and optimization into an end end-to-end framework. However, computing the decision loss typically involves solving optimization problems with non-differentiable operations, making difficult for automatic differentiation tools in machine learning frameworks such as PyTorch [26] and TensorFlow [ to provide correct gradients. Prior work has proposed three main strategies for gradient computation: (1) differentiating optimality conditions (e.g., via KKT or self-dual formulations, as in OptNet 4. DQP 122 QPTL. 42). and IntOpt (1)), (2) smoothing by
random perturbations and treating the optimization as black box (c.g., DBB (7), DPO Z. I-MLE (25)), and (3) using surrogate loss functions (e.g.. SPO [1), LTR [22). LODL (2). TaskMet(6). Lancer (50)). The first approach is limited to convex quadratic or linear programs, which do not fit settings of resource allocation problems. The second, while more general, is computationally expensive and impractical for large-scale marketing data. The third relies on access to optimal solutions, which are typically unobservable in offline marketing scenarios due to counterfactuals. As a result, effectively applying DFL to real-world marketing resource allocation remains challenging.
We emphasize that although existing DFL methods can address the inconsistency between prediction and decision objectives, none can be directly applied to marketing optimization due to domain-specific challenges such as the multi-choice knapsack problem, constraint uncertainty, counterfactuals, and the computational demands of large-scale datasets. Therefore, the most relevant works to ours are two DFL applications in marketing: DHCL (51] and DFCL [52]. DHCL directly learns unbiased estimator of the decision factor in OR by customized loss, while DFCL introduces two surrogate losses (DFCL-DPL and DFCL-DIFD) for effective gradient estimation of the dual decision loss within the DFL paradigm. However, both approaches still have two notable limitations:
Exacerbation of the bias-variance dilemma. In DHCL and DFCL, counterfactuals prevent direct computation of decision loss, so it can only be unbiasedly estimated from RCT data. Thus, abundant OBS data cannot be used for training, and learning is limited to scarce RCT samples, making models prone to overfitting and poor generalization (low bias but high variance).
Insufficiency in addressing prediction-decision misalignment. DFCL still faces two key issues in aligning prediction and decision objectives. First, its loss is a weighted sum of decision and prediction losses, with the trade-off controlled by manually tuned hyperparameter a, which inflexible and not fully automated. Second, DFCL uses a dual decision loss that evaluates quality
3

---

across all possible budgets, while real-world marketing budgets are typically limited to narrow or discrete set. This mismatch can reduce alignment with actual decision quality in practice.
3 Problem Formulation
We initiate our formal analysis with a marketing optimization scenario involving M distinct treatments. For each mntaiiuualtreatmmentt pair (i, let rij E IR+ and Cij IR+ denote the potential revenue and associated cost respectively. The constrained optimization objective requires developing an allocation policy [N] [M] that maximizes the platform's 's cumulative revenue under global budget constraint B. This combinatorial decision- decision-making making challenge. which we term the Toldi-Treatment Budget Allocation Problem (MTBAP), admits the following primal and dual formulations:
max H(z ΣΣ> Tig
i 1
AB + ΣΣ (nyy Acty)22s
max
s.t. ΣΣ 25y1 SB
min X20
s.t. Σ Fij 1, ViE [N]
Σ 3i 5= = ViE [N]
Fij (0, 1J, Vi E j [M]
Eg (0, 1), V
(N], E E [M]
Figure 1: The primal (left) and dual (right) formulations of the MTBAP.
The binary variable Zzy f0, 1} indicates whether individual is assigned treatment j. The primal problem is an instance of the NP-Hard MCKP [37] The Lagrangian relaxation algorithm A (see
Appendix I. efficiently finds the optimal solution to dual problem via binary search for X* yielding an approximate solution to primal problem with a worst-case approximation ratio of p 1 OPT
dax faz
z A(H())), ((o arg max Tr TTy--aacy -xarr J'EM
(I)
where 1 is indicator function. Let 0 denote the parameters of Target Network Fo with f(0) and a(0) representing the predicted revenue and cost for individuals under different treatments, respectively.
The prediction loss LP (9) is defined as the following MSE Loss between predicted and true values:
C EE JE[M] [(ri - (0) (cy (0))
Given predicted parameters f() and 6(0), the allocation policy 2" (f(0), is obtained by applying algorithm A to the optimization problem (=; r(0), ē(O)), shown in eq and Appendix A. The decision loss LDL directly quantifies decision quality through the negative realized objective value:
(0)( M M EseIN) [ (F(O), a(0) -riy]
Note that the prediction loss LpI. enhances model generalizability by minimizing estimation errors, whereas the decision loss LDI. evaluates policy suboptimality in downstream OR tasks and enables real-time decision quality awareness of the model. Thus, the composite objective LDFCI in DFCL [22 is formulated to explicitly captures the dual objectives predictive accuracy and decision quality as:
CDCCL LDL aLPL
In digital marketing causal inference, each sample is represented by (X, R, C), where x; denotes user features, t; the assigned treatment index, and (Tze, Cas,) the observed factual revenue-cost pair under Rubin's potential outcomes framework [1]] The complete counterfactual surfaces (R(t), Ctt) remain partially observable across two distinct data modalities: experimental data DReT from randomized controlled trials satisfies strong ignorability (X, R(t), C(t)) 1 T yet suffers from prohibitive collection costs and scarcity, whereas observational data DoBS provides abundant samples via passive collection the expense of confounding biases due non-random treatment assignment.
The fundamental challenge in causal inference originates from Rubin's missing counterfactual problem: for any individual exposed to treatment 4, only the factual outcome (rit, Cit,) ) is observed, while the counterfactual responses {[rij, Gj))ive, remain fundamentally unobserved. This inherent data incompleteness implies the ground-truth values (r) can never fully ascertained, making both prediction loss LPL and decision loss CDL non-computable given either DRCT DoBs.

---

Lower-level Optimization
Implisit Diffrrnntiaionn
Upperlevel Optimization
Ausureyy
Targal
Molded FP
Quality
Prackdion Lowe Arclou&:Pass)
Cipy 2
(1P-4) - Alll 83)
2106
Dedision Loss Are. (o)()); Phurr)
PLPY) EEG)
Updiél Eleps
2C
Cheeroed
Countee de
Model Pred osion F
Uncineed Fitimation
Dons
DACT
Obseraaioonl Data
Bindge Network Up
Eepertmental
Figure 2: Overview of the Bi-Level Decision-Focused Causal Learning (Bi-DFCL) Framework.
4 Proposed Methods
4.1 Bi-level Optimization Framework
As discussed in Sec. our objective is to minimize the composite loss LDFCI. = LDL + aLpr.- While the DFCL framework (52) minimizes this loss solely on DRCT to ensure unbiasedness, this approach overlooks complementary strengths of both DRCT and DoBS. as well distinct advantages of LDL and Lri. Specifically, LDL is highly dependent on the unbiasedness of 'DCCT: minimizing CDL on biased DOBS would greatly amplify bias and severely degrade decision quality, Conversely, CPL is designed to improve generalization and is most effective when optimized on low-variance, large-scale DoBS. Ultimately, our goal is for Target Network Fg trained on DOBS to achieve high decision quality on DRCT. Motivated by this, we propose applying LDL to DRCT and LPL to DoBS, assigning them to the upper and lower levels of bi-level optimization framework, respectively.
e- arg min LDL (0' (Ø); DRCT)
s.t. 0r (6) arg min Cr 0: DOBS).
(6)
f and denote parameters of Target Network Fo and Bridge Network Gø. respectively. This setup constitutes a bi-level optimization (BI.)) problem 46
20), where the upper-level (5) and the lower-level (6) are nested: the objective and variables of upper level depend on the optimizer of lower level. The core idea is to end-to-end learn Go by minimizing LDL on DRCT, such that the parameterized prediction loss (ó, 0) on DoBS is adaptively refined. The bridge vector w, output by Go. is dynamically updated and used to generate counterfactual pseudo-labels ret C on DoBS:
15 28
r = ( (v) wh, + (0) (1 w(gg)
G = (v) wf, + (.)( (1 wf,)), w sigmoid(( wh,-- (1
(7)
(8)
Here, w acts as a gating coefficient, adaptively combining outputs from Fg and fixed teacher Network Fg (pretrained on DROT via any uplift model and kept fixed). This mechanism bridges DOBS and DRCT and generates stable counterfactual pseudo-labels to parameterize L.L on DoBs:
CPL (0, = E,,,, (Tw, - )² 2 + (ca, - ) + Ei,i ((rif - f.g)² + (c - 2g)
By fully leveraging unbiased decision signals from DRCT, this approach makes the lower-level (6) both decision-aware and less biased, dynamically correcting the learning direction Target Network Fo. Assigning LDI, and LPI. to the upper and lower levels also enables adaptive balancing of two learning objectives LDDCI. = LDL. + aLLr,, thus eliminating the need for manual hyperparameter tuning of a. An overview of the Bi-DFCL framework is shown in Figure 2 Despite these advantages, solving the resulting bi-level optimization problem is non-trivial. The lower-level loss (6) is differen- tiable with respect to 0, allowing Fe to be updated via gradient descent (GD). However, computing the gradient for the upper-level loss (5) is much more challenging. By the chain rule, we have:
(
VoLoL (0* (o): VoLDL (0; DRCT (RCT))l==+<o) (6))
(10)
To calculate the gradient V VoL (0); PRCT), we require both VeLDL DRCT) at 0 = 0 (d)
and Jacobian a0" (0) However, as will be discussed in See 4.2 LDL is non-differentiable, and thus
5

---

the first term cannot be directly computed. Moreover, the second term is also difficult to obtain, as the optimal solution 0- (Ø) lacks closed-form expression. making Jacobian intractable. We will discuss how to address these two non-differentiabliit challenges in Sec, 4.2 and Sec. 4.4 respectively.
4.2 Differentiation of Decision Loss
As is mentioned in Sec. 3 CDL is non-computed due the lack of the counterfactual responses. By leveraging strong ignorability (X, R(t), C(t) 1 I of experimental data, we derive an unbiased estimator of the decision loss as follows (see Appendix A.2 the formal proof):
N Nr,
LD. (0; DRCT) = -Biie
" (f(0),
rit.
(11)
Nr. is the number of individuals assigned treatment 1 DRCT, and by the chain rule, the gradient is:
VoLDL (RCT) (C DRCT) dz (f(0).
(12)
(f()),
00
The first term is trivial since LDL continuously differentiable with respect 2 (f(0), according to Eq. 11 Based on the Lagrangian relaxation algorithm A (T) the solution is:
*.. (F(0), = { = arg max (F) X xaa (0,10) JEIM]
(13)
where 1 is indicator function and X is the optimal Lagrange multiplier. By introducing dual decision variables 2 (f(0), (()) satisfying o (f()), é(0) = 11, marg max (0) (0) the Lagrange
multiplier X in Eq. (33) can be determined by binary search of X with the terminal condition:
N N
B N
E4,t
(f(0), Cita
S.
(14)
Due to the existence of indicator functions, 2 (f(0), (()) is non non-differentiable with respect to 6. By utilizing Softmax functions, the discrete solution , (f(0), e(0)) can be relaxed to continuously differentiable function 2/2, (f(0), E())). which can also be regarded as the probability of zi:: 1:
1,h ((()) = explfir, (0) - X Cr (0)
(15)
EjE[M] exp F (0) X (x)s(())) (0)
Hence, we obtain surrogate decision loss LrPl of LDL, called the primal policy learning loss:
N
explfir, (0) X Cu, (0)
L0p DRCT) = -Ei,
No.
JjEan expl - (0)
Tit,
(16)
Note that minimizing LPPL (0: DRCT) is equivalent maximizing the expected reward of policy
π = = (f(0), ((0)). Additionally, an alternative derivation the primal policy learning loss can be obtained through the maximum entropy regularization trick, detailed in Appendix A.3 Unlike dual decision loss in [52) which considers all budgets, LPPI. directly targets decision quality under a specific budget B, thereby ensuring better alignment with real-world marketing constraints.
We further introduce the primal improved finite difference strategy (PIFD), which leverages the mathematical definition of the gradient terms (Cpr.(:PPCTT) PIFD directly estimates their values via
(F((((())_(00)
black-box perturbations on LDL, and accelerates computation with Lprr.-aware gradient estimator (see Appendix A.4 for details). Compared to LPPI., PIFD preserves original optimization landscape without relaxation, and by freezing computed gradients as non-trainable nodes, enables seamless integration with automatic differentiation libraries. This final surrogate decision loss LPIED is given:
(Dv DacT). (F(, ((0))
( DRCT) = [EE(M)JE[M]
(,(((0),
(17)

---

4.3 Implicit Differentiaaion--ass Algorithm
Next, we address the second challenge in Bi-DFCL: computing the Jacobian (6"(6) de without closed-
form solution for 8* (Ø), well-known issue BLO. common approach explicitly differentiate through the gradient descent step, assuming 8 (e) can be reached in one GD step [15 9 41 (see Appendix A.5). However, this method relies on the optimization path and, when combined with decision loss, often suffers from vanishing gradients and suboptimal solutions. To address this, we propose an implicit differentiation-bann based algorithm. Note that the optimal solution 0* (() satisfies the
first-order condition:
10 (0) (+) = O. Differentiating both sides with respect to @ gives:
80
8 LPL 0; DOBS)
(6) 0 CoL(o, 0: DOBS)
002
(0)
aΦ
0o00
(=-0(a)
(18)
Eq. (18) is also a direct result of the implicit function theorem [34. Notably, this approach avoids explicitly storing the optimization trajectory; the optimal solution 0* (1) can be obtained using any
optimization algorithm, and we only need to differentiate the optimality condition satisfies to implicitly obtain its Jacobian. This path-independence leads to more accurate and stable gradients.
While a closed-form expression for the Jacobian (d"(() can be directly derived, computing and storing the inverse of the Hessian matrix is computationally expensive, especially in large-scale marketing applications. To overcome this, we employ the conjugate gradient (CG) algorithm [34), which solves Az = b by equivalently minimizing )= Ax bb x and can be implemented using only Hessian- vector products. This approach efficiently solves (18) without explicit Hessian construction or inversion (see Appendix ) making Bi-DFCL applicable large-scale marketing optimization.
4.4 Overall training procedure of Bi-DFCL
We now summarize the overall training procedure of Bi-DFCL in Algorithm
Algorithm Pseudocode for Bi-Level Decision-Focused Causal Learning (Bi-DFCL)
Input:
DACT [[I:;, li, Tie,, (x) NECT,, DoBs [(1; 4, Yay, Cat, )) Nons, Target Network Fe. Bridge Network Go. Teacher Network F (number of GD steps for assumed updates, default k: 5). Pretrain Teacher Network F on DRCT using any uplift model with standard MSE loss
Initialize Target Network Fe (random or warm start) and Bridge Network Go (random).
1: for each mini-batch B/ in DOBS over all epochs do
2: if b mod k = 0 (i.e., every k-th batch), then solve the upper-level problem (5)
3: Step 1 Perform k assumed updates to obtain 0* (Ø) (without modifying Fe):
4:
Copy Fo to Fú; generate counterfactual pseudo-labels of G. for B8 as in Eq.
8
5:
Perform & steps gradient descent((DD [r, 0; (6 (Eq. (9) that Fg update Fé.
Step 2- Oblain two non-differentiabiliy terms as shown Eq. (10):
7:
Solve Eq. (18) via conjugate gradient (CG) Algorithm obtain Jacobian
(0)
de Using Fo,, compute Crri 16 or LriFE 17 on DRCT obtain yp (f; DRCT) (am0-
8:
9:
Step 3 - End-to-End update Bridge Network Go according TO:
10:
Perform one GD step on Ge with DRCT <- cxp
VaLra (0; DRCT) PROT))==000)) (0)
00 (d)
11:
end if
12:
Solve the lower-level problem (6) with the latest Go:
13:
Generate updated counterfactual pseudo-labels rit, city for Bol as in Eq. 7 8
14:
Compute LPt (ó, 0; ((())) (Bq. 9 update Fo by one GD step: 0 -aa VoLPI. (0, DoBs).
15:
end for
Output: Well-trained Target Network Fo for predicting fiy. &gz.
5
Real-World Experiments
5.1 Offline Experimental Setup
Dataset and Preprocessing. Three types of offline datasets are provided: an open real-world dataset and two marketing datasets collected from Meituan, an online food delivery platform. The detailed statistics of three datasets are shown in Table [ Readers see more details in Appendix B.]
7

---

This public dataset from Criteo [T] contains 13.9 million RCT samples, each with 12 features, a binary treatment indicator, and two response labels (visit/conversion), Since practical marketing scenarios typically have large number of OBS data and little RCT data, we simulate a marketing policy to convert part of the RCT data into OBS data. Further details can be found in Appendix [BI.I] We refer to the transformed dataset as CRITEO-UPLIFT v2 (Hybrid).
CRITEO-UPLIFT v2.
Marketing data I. Money-off is a common marketing campaign at Meituan, an online food delivery platform. We conduct two-month RCT collect data in this platform. The money-off T E {0, ......77 } is taken as the treatment, where T = t means St cash off for each order whose price meets given threshold. This dataset contains 180 features, I treatment label and response labels (daily cost/orders). This dataset contains 5.5 million RCT and 22.2 million OBS samples.
Marketing data II. Discounting is another common marketing campaign at Meituan. We conduct a four-week RCT to collect data. The discount T {0, 5, 10, 15, 20} is taken the treatment, where T = t means 5% off for each order whose price meets a given threshold. This dataset contains 192 features, treatment label and 2 response labels (daily cost/orders). This dataset contains 5.0 million RCT samples and 33.8 million OBS samples.
Table 1: Statistics of three offline datasets.
Dataset
Features
Treatment
Training (OBS)
Training (RCT)
Validation (RCT)
Test (RCT)
CRITEO-UPLIFT v2 (Hybrid)
12
2
3498294
698980
1397959
4193878
Marketing data
180
8
22201405
2220781
555014
2775976
Marketing duta
192
5
33815274
2017450
504362
2521813
Baselines and Experimental Details. We compare the proposed methods with three categories of causal learning baselines: (1) Methods trained with RCT data, (2) Methods trained with OBS data, and (3) Methods trained with both RCT and OBS data. Also see more details in Appendix 1.3
Methods trained with RCT data: With RCT data only, the baselines include two simple two-stage methods: TSM-SI. 48]. TSM-CF(), and three end-to-end methods: [HCI[11], DFCL-DPL [52.
(FCL- Note that these end-to-end methods can only be trained using RCT data.
Methods trained with OBS data: With OBS data only, the baselines include two simple two-stage methods: TSM-SL(48) TSM-CF(). and two reweighting-basee methods: IPS 30), 4R-JT(40), and three representation learning methods: CFR-WASS 333, CFR-MMD 33), DragonNet(35).
Methods trained with both RCT and OBS data: Based on both RCT and OBS data, the baselines include TSM-SL(48) and reweighting-beat based methods: LTD-IPS [41, LTD-DR 44], AutoDebias[9),
and representation learning methods: CausE g KD-Label
[9] KD-Feature 19
Evaluation Metrics.
Two evaluation metrics are provided for offline evaluation in this experiment.
AUCC (Area under Cost Curve). A common metric used in existing works 2 13 which is designed for evaluating the performance to rank ROI of individuals in the binary treatment setting. Because AUCC represents the decision quality of marketing under binary treatments, we use AUCC v2(Hybrid).
to compare the performance of different methods CRITEO-
EOM (Expected Outcome Metric). EOM is also commonly used in RCT data, an unbiased estimation of the expected outcome (per-capita revenue/per-capee Based on cost) for arbitrary budget allocation policy can be obtained. Details of EOM are shown in Appendix B.2 Since EOM represents the decision quality of marketing under multipple treatments, we use EOM to compare the performance of different methods in Marketing data and II.
49 52
5.2 Offliin Experimental Results
Overall Performance Comparison. Table compares Bi-DFCL with all baselines. We have four main observations: (1): Among methods trained solely on RCT data, end-to-end methods consistently outperform two-stage methods across all datasets, highlighting the importance of directly optimizing for decision quality and validating our motivation to bridge the prediction-decision gap. (2): Our proposed DFCL-PPL. and DFCL-PIFD outperform dual decision loss, showing that optimizing primal decision losses better aligns with real-world marketing constraints, as they directly target decision quality under specific budget values B. (3): The relative performance of TSM trained on RCT or
8

---

Table 2: Performances of the proposed methods and baselines (mean + standard deviation across 20
runs). The best result is bolded and the best results of three types of baseline methods underlined.
EPLIET (lybrrdd)
Markeeingg Data 1
Marketing Data
Data
Methods
AUCC
Improvement
BOM
Improvencct
EOM
Improvemennt
RCT
TSM-SL
0.7149:40.0299
1.00000000022
00000
RCT
TSM-CF
0.6730:10.0196
-5.78%
0.9767=0000005
-2.33%
0.0660:000006
-3.20%
RCT
DHCL
1.00%
0.9972:5000111
-0.28%
1. 10059:00 0007
0.5995
RCT
DFCL-DPL
0.7416:2:000170
3.82%
0.0120=00.0200
1.20%
1.0004::00 000S
0.945
RCT
DECI-DIFD
0.744 223
4.17%
10151=000033
1.51%
1.01 10:1000009
1. 1.00%
RCT
DECL-PPL (Ours)
0,7119:10.0122
3,86%
1.016750.002
1.67%
1.0156::000006
1,56/2
RCT
DFCL-PIED (Ours)
0.7427:40.0244
4.12%
1.120000.024
1.20%
00015
OBS
TSM-SL
0,7413:40.0008
3.78%
1.0067
0.67%
0,9957 +000011
-043%
OBS
TSM-CF
0.7105-:0.0000
-0.53%
0.0022
-1.75%
0,9800000
-3.00%
OBS
IPS
0.70922:000111
-0.71%
00000
0.70%
0,9990:20 0002
-0.10%
ORS
DR-JT
0. 74055000053
4.14%
L.002 9019
1.02%
1 10054 0018
0.54%
OBS
CFR-WASS
0.7245:10.0009
1.43%%
1.0032:0.0000
0039%
OBS
CFR MTR
0.7339:10.0045
2.74%
1.0055
0.55%
0,9997:1000033
0.03%
OBS
DragonNet
0.7000:00.0066
1.66%
1,009 0.0004
0.69%
0.9988.10.002
0.12%
RCT:0BS
TSM-SL
0.7498::0.0022
4.13%
1.0071 0000011
0.71%
0.998--0000022
4012%
RCT+OBS
CausE
0.7392:+00008
3.49%
10031 0000009
0.31%
1.0001 #000014
0.01%
RCT+OBS
KDD-ahel
0.73744:100055
3.23%
0002
0.9997
4100%
RCT+OBS
KD-Foatume
0.7306:10.0061
2.28%
1.004
0.74%
0,9983 0009
40.17%
RCT40BS
LTD--PS
0.7427 0.0080
3.98%
1.1 0012002000036
1.209
1.0040:1000042
0 0.40%
RCT+0BS
LTD DDD-DR
0.7533-4: 0.0059
5.16%
1.016 0026
1.68%
1,0067 0000001
0.67%
RCT+OBS
AmtoDehins
0.7489::
4.84%
L.075
1.25%
L0066-+0
RCT+OBS
BI-DECL-PPL (Ours))
0.7797:1: 0.0094
9.16%%
1,0277 0.0.0024
2.77%
1.0252-10.0023
2.52%
RCT1OBS
BI-DPCL-PIED (Ours)
0.7812-t: 0.0084
9
000000
2,97
1.044::0000008
2.49%
OBS data varies across datasets, reflecting the complementary strengths of the two data sources: RCT data offer low bias but high variance, while OBS data are more biased but lower variance. However, all existing end-to-end methods are restricted to RCT data, limiting their ability to leverage abundant OBS data and making them prone overfitting. (4): Bi-DFCL consistently outperforms all baselines on all datasets, demonstrating superior alignment of prediction and decision objectives and ability to achieve optimal bias-variance tradeoff by fully leveraging both RCT and OBS data. By overcoming the overreliance on limited RCT data that hampers previous recision-focuse methods, Bi-DFCL delivers improved generalization and decision quality in real-world marketing scenarios.
Ablation Studies. To show the effects of individual components, we conduct ablation study by incrementally adding four key components of Bi-DFCL to baseline in sequential manner: Decision Loss (PPL), Bi-level Optimization by hybrid RCT and OBS data, Counterfactual Labels, and Implicit Differentiation Algorithm. The experimental results on marketing datasets are reported in Table We can find that after the introduction of each module, the performance can all be strengthened to some extent, which demonstrates that our three contributions can all benefit the marketing optimization. In addition, we provide detailed descriptions for these baselines of ablation studies in Appendix [B.2 BB.2
Table 3: Ablation study of each individual component in Bi-DFCL with two marketing datasets.
Components Di-DFCL
Marketing Data I
Marketing Data
Decision Loss (PPL) Bi-level Optimization
Counerfactual Labels
Implicis Dificrentiaion
HOM
Improvement
BOM
Improvement
x 1.0000
L.0000
1,0167
1.67%
L0156
1.56%
1.0240
2.40%
1.0175
1.75%
X 1.0248
2.48%
1,0213
2.13%
1.0277
2.77%
1.0252
2.52%
In-depth Analysis. We conduct in-depth analysis to explore the effect of the training data size, as well as to validate the bias-variance properties of the RCT and OBS data. We further evaluate the sensitivity of the proposed methods under multiple sets of budget values B. Additionally, we also provide a detailed discussion comparing the computational overhead of Bi-DFCL against different baseline methods. See Appendix B.3 for more detailed experimental results.
using different values and evaluate the robustness of our
5.3 Online A/B Tests
Setups. We deploy our proposed Bi-DFCL-PPL, Bi-DFCL-PIFD and three baselines: DFCL-PIFD, L.TD-DR and TSM-SI. together to support a discount campaign at Meituan (our online food delivery platform) and conduct large-scale online A/B tests for four weeks. The experiment contains 790K

---

online shops and they are randomly divided every day into five groups called G-BPPL, G-BPIFD, G- PIFD, G-LTD and G-TSL respectively. Each shop will be assigned discount t {0, 5, 10, 15, 20) as the treatment, which means 1% off for each order whose price meets given threshold. The marketing goal is to maximize the orders by assigning an appropriate discount to each store every day for a limited budget that may change slightly from day to day.
Table 4: Results of online A/B with the confidence interval (four weeks)
Week
Method
Group
Improvement
1st
2nd
3rd
4th
TSM-SL
G-TSL
1.0000 + 0.0022
1,0335 1 0.0030
0.9217 + 0.0017
0,9720 =: 0,0048
LID-DR
G-LTD
1.0183 +: 0.0020
1.0378 + 0.0039
0.9314 + 0.0037
0.9723 = 0.0070
0.91%
DFCL-PIFD
G-PIFD
1,0302 +: 0.0013
1.0436
0.0020
0.9440
0.9799
0.0018
1.80%
Bi-DFCL-PPL
G-BPPL
1.0128 4: 0.0019
1.0558
0.0025
0.9582
0.0019
0.9872
0.0014
3.00%
Bi-DFCL-PIRD
G-BPIED
1.0470 + 0.0021
1.0537 0.0027
0.9581
0.0024
0.9906 0.0031
3.22%
Results. Table 4 illustrates the online weekly orders for five groups during four weeks. In order to
preserve data privacy, all data points have been normalized that are divided by the orders of TSM-SL the first week. We can see that Bi-DFCL exhibits significantly superior overall performance during four weeks, which validates the effectiveness of Bi-DFCL for real-world marketing optimization.
6 Conclusion
In this paper, we propose the Bi-Level Decision-Focused Causal Learning (Bi-DFCL) framework for large-scale marketing optimization, addressing two key challenges in existing approaches: prediction- decision misalignment and bias variance dilemma. Extensive offline experiments and online A/B tests demonstrate that Bi-DFCL consistently outperforms state-of-the- thr-ar. Our future work includes further improving computational efficiency and applying Bi-DFCL to other didision-making domains.
Acknowledgments
This work was supported in part by the NSF China (62422207).
References
[1] M. Abadi, A. Agarwal, Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, et al. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint
2016.
[2] M. Ai, B. Li, H. Gong, Q. Yu, S. Xue, Y. Zhang, Y. Zhang and P. Jiang. Lbef: A large-scale budget-constrained causal forest algorithm. Proceedings of the ACM Web Conference 2022, pages 2010-2319, 2022.
[3] J. Albert and D. Goldenberg. E-commerce promotions personalization via online multiple- choice knapsack with uplift modeling. Proceedings of the 31 ACM International Conference on Information & Knowledge Management, pages
2022.
[4] B. Amos and J. Z. Kolter. Optnet: Differentiable optimization layer in neural networks, In International Conference on Machine Learning, pages 136-145. PMLR, 2017.
[5] S. Athey, J. Tibshirani, and S. Wager. Generalized random forests. 2019.
[6] D. Bansal, R. T. Chen, M. Mukadam, and B. Amos. Taskmet: Task-driven metric learning for model learning. Advances in Neural Information Processing Systems, 36, 2024.
[7] Q Berthet, M. Blondel, O. Teboul, M. Cuturi, J.-P. Vert, and F. Bach. Learning with differen- tiable pertubed optimizers, Advances in neural information processing systems, 33:9508-9513 2020.
[8] S. Bonner and F. Vasile. Causal embeddings for recommendation. In RecSys, 2018
10

---

[9] J. Chen, H. Dong, Y. Qiu, X. He. X. Xin, L. Chen, G. Lin, and K. Yang. Autodebias: Learning debias for recommendation. In SIGIR, 2021.
[10] A. Deaton and N. Cartwright. Understanding and misunderstanding randomized controlled trials. Social science & medicine, 210:2-21, 2018.
[11] E. Diemert, A. Betlei, C. Renaudin, and M.-R. Amini. large scale benchmark for uplift modeling. In KDD, 2018.
[12] Donti, B. Amos, and Z. Kolter. Task-based end-to-end model learning in stochastic optimization. Advances in neural information processing systems, 30, 2017.
[13] Du, Lee, and Ghaffarizadeh. Improve user retention with causal learning. In The 2019 ACM SIGKDD Workshop Causal Discovery, pages 34 49. PMLR, 2019.
[14] A. N. Elmachtoub and P. Grigas. Smart "predict, then optimize". Management Science, 68(1): 9-26, 2022.
[15] C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. International conference on machine learning, pages 1126-1135. PMLR, 2017.
[16] F. Johansson, U. Shalit, and D. Sontag. Learning representations for counterfactual inference. In International conference on machine learning, pages 3020-3029. PMLR, 2016.
[17] S. R. Künzel, J. S. Sekhon, P. J. Bickel, and B. Yu. Metaleamnrrs for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences, 116 (10):41564415,, 2019.
[18] H. Li, Y. Xiao, C. Zheng, and Wu. Balancing unobserved confounding with a few unbiased ratings in debiased recommendations. In Proceedings of the ACM Web Conference 2023, pages 1305-1313, 2023.
[19] D. Liu, P. Cheng, Z. Dong, X. He, W. Pan, and Z. Ming. A general knowledge distillation framework for counterfactual recommendation via uniform data. In SIGIR, 2020.
[20] R. Liu, Y. Liu, W. Yao, S. Zeng, and J. Zhang. Averaged method of multipliers for bi-level optimization without lower-level strong convexity. In International Conference Machine Learning, pages 21839-21888 2166. PMLR, 2023.
[21] J. Mandi and T. Guns. Interior point solving for Ip Ip-based prediction+ optimisation. Advances in Neural Information Processing Systems, 33:7272-7282, 2020.
[22] J. Mandi, V. Bucarey, M. M. K. Tchomba, and T. Guns. Decision-focused learning: through the lens of learning to rank. International Conference on Machine Learning, pages 14935-14947. PMLR, 2022.
[23] Mandi, Kotary, S. Berden, M. Mulamba, Bucarey, T. Guns, and F. Fioretto. Decision- focused learning: Foundations, state of the art, benchmark and future opportunities. arXiv preprint arXiv:2307. 13565, 2023.
[24] X. Nie and S. Wager. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika. 108(2):299-315 2021.
[25) M. Niepert, P. Minervini, and I.. Franceschi. Implicit mle: backpropagating through discrete exponential family distributions. Advances in Neural Information Processing Systems, 34: 14567-14579, 2021.
[26] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019.
[27] M. Pogančić, Paulus, V. Musil, G. Martius, and M. Rolinek. Differentiation of blackbox combinatorial solvers. In International Conference Learning Representations, 2019.

---

[28] A. Rajeswaran, C. Finn, S. M. Kakade, and S. Levine. Meta-learning with implicit gradients. Advances neural information processing systems, 32, 2019.
[29] U. Sadana, A. Chenreddy, E. Delage, A. Forel, E. Frejinger, and T. Vidal. A survey of contextual optimization methods for decision-making under uncertainty. European Journal of Operational Research, 320(2)271--88) 2025.
[30] T. Schnabel, A. Swaminathan, A. Singh, N. Chandak, and T. Joachims. Recommendations as treatments: Debiasing learning and evaluation. In International Conference on Machine Learning, 2016.
[31] S. Sekhon. The neyman-rubin model causal inference and estimation via matching methods. The Oxford Handbook of Political Methodology, 2:1-32, 2008
[32] S. Shah, K. Wang, B. Wilder, A. Perrault, and M. Tambe. Decision-focused learning without decision-making Learning locally optimized decision losses. Advances in Neural Information Processing Systems, 35:1320-1333 2022.
[33] U. Shalit, F. D. Johansson, and D. Sontag. Estimating individual treatment effect: generalization bounds and algorithms. In International conference on machine learning, pages 3076-3085. PMLR, 2017.
[34] J. R. Shewchuk et al. An introduction to the conjugate gradient method without the agonizing pain. 1994.
[35] C. Shi, D. Blei, and V. Veitch. Adapting neural networks for the estimation of treatment effects. Advances Neural Information Processing Systems (NIPS), 32, 2019.
[36] B. Sibbald and M. Roland. Understanding controlled trials. why are randomised controlled trials important? BMJ: British Medical Journal, 3 221)), 1998.
[37] Sinha and A. A. Zoltners. The multiple-choice knapsack problem. Operations Research, 27 (3):503-515 1979.
[38] S. Wager and Athey. Estimation and inference of heterogeneous treatment effects using random forests. Journal of the American Statistical Association, 112652)):122--122,, 2018.
[39] C. Wang, X. Shi, S. Xu, Z. Wang, Z. Fan, Y. Feng, You, and Y. Chen. A multi-stage framework for online bonus allocation based on constrained user intent detection. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5028-5038. 2023.
[40] X. Wang, R. Zhang, Y. Sun, and J. Qi. Doubly robust joint learning for recommendation on data missing not at random. In International Conference on Machine Learning, 2019.
[41] X. Wang, R. Zhang, Y. Sun, and J. Qi. Combating selection biases in recommender systems with a few unbiased ratings. In WSDM, 2021.
[42] B. Wilder, B. Dilkina, and M. Tambe. Melding the data-decisions pipeline: Decision-focuse learning for combinatorial optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1658-1665, 2019.
[43] L. Yao, S. Li, Y. Li, M. Huai, J. Gao, and A. Zhang. Representation learning for treatment effect estimation from observational data. Advances in neural information processing systems, 31, 2018.
[44] Ye, J. Qian, J. Chen, C.-h. Wu. Y. Zhou, S. De Mars, F. Yang, and L. Zhang. Customized regression model for airbnb dynamic pricing. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pages 932-940, 2018
[45] Y. Zhang, B. Tang, Q. Yang, D. An, H. Tang, C. Xi, X. Li, and F. Xiong. Bcorle (A): An offline reinforcement learning and evaluation framework for coupons allocation in e-commerce market. Advances in Neural Information Processing Systems, 34:20410-200 10-222,, 2021.
12

---

[46] Y. Zhang, P. Khanduri, I. Tsaknakis. Y. Yao, M. Hong, and S. Liu. An introduction to bi-level optimization: Foundations and applications in signal processing and machine learning, 2023.
[47] H. Zhao, Q. Cui, X. Li, R. Bao, L. Li, J. Zhou, Z. Liu, and Feng. Mdi: debiasing method combining unbiased and biased data. In SIGIR, pages 3280-3284, 2023.
[48] K. Zhao, J. Hua, L. Yan, Q. Zhang, H. Xu, and C. Yang. A unified framework for marketing budget allocation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1820-1830. 2019.
[49] Y. Zhao, X. Fang, and D. Simchi-Levi. Uplift modeling with multiple treatments and general response types. In Proceedings of the 2017 SIAM International Conference on Data Mining, pages 588-596. SIAM, 2017.
[50] A. Zharmagambetov, B. Amos, A. Ferber, Huang, B. Dilkina, and Y. Tian. Landscape surrogate: Learning decision losses for mathematical optimization under partial information. Advances Neural Information Processing Systems, 36, 2024.
[51] H. Zhou, Li, G. Jiang. Zheng, and D. Wang. Direct heterogeneous causal learning for resouree allocation problems in marketing, In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 5446-5454, 2023.
[52] H. Zhou, R. Huang, S. Li, G. Jiang. J. Zheng, B. Cheng, and W. Lin. Decision focused causal learning for direct counterfactual marketing optimization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2368-6379, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction include the claims made in the paper, and our contributions are clearly summarized in sec. l:introduction..
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of the work in sec. 6: Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

The paper provides the full set of assumptions and complete proofs for the theoretical results in the main paper and the SpppeniixSupplementtal material).
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper fully discloses all the information needed to reproduce the main experimental results and Experimental Details in Sec. 4 and 5 and Appendix(Suupplemnnta
material).
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
15

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We provide open access the public dataset and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nnis.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.co. cc/ppiccc guwdes/ddeeeee CooeeSumeeeeeeeeeeeeee ssi onyylicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper specifies all the training and test information including datasets, preprocessing, experimental protocols and details in Sec. 5 and Appendix(Supplemmnnal material).
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We clearly report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments in sec. 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides sufficient information on the computer resources (type of compute workers, memory) in Appendix(SSpplementaa material).
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't fidn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.c http:////eeeeeeecccppp/////ciiea insu
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss potential positive socictal impacts abstract, introduction and acc.5.:online a/b tests and Appendix(Supplemnntaa material).
Guidelines:
17

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negatíve societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The creator or original owner of the assets (e.g., code, data, models) used in the paper is properly credited, and the license and terms of use are explicitly mentioned and appropriately respected.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, datasets.paperssiii paperswi thcode. www.com//aaaa datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: [Yes]
Guidelines: Our code is available(see abstract) and the documentation is provided alongside.
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Large language models (LLMs) are not used as an important, original, or non-standard component of the methods in this work.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurl)) attt:://eeaarrg..cc//ooofffreeer ccC///ofeeeeeeese/222////.. for what should or should not be described.
20

---

A
More details of Bi-DFCL Framework
Here, we provide additional details for Sec.3 and Sec.4 of the main text.
A.1 The Lagrangian relaxation algorithm A in Sec.3
We give pseudocode of the Lagrangian relaxation algorithm in Algorithm 2
Algorithm The Lagrangian relaxation algorithm for the primal formulation MТBAP
Input: budget B; predicted revenue/cost ê, é; data D = {1 4, Rit, Cit, }) Ti=1 small constant c. Compute: For each in [M]. N2 <- number samples with 1, t; Pt 4- N//N.
Initialize: Amin <- 0, Amax <- maxij
fsg Ees
Pij 0 for all i,j
1: while Amax Amin > e do Amar +mmi
2:
3: for all j do
4: 2gj f I(j = arg maxy (fig Ad,5))
5:
end for
6: a(,,r,,, t + E. L a. (t: = arg max, Zij)
7: if B T,,,T.C.f f, < then
8:
break
9:
end
10:
if N
c(A,r, > 0 then
11:
Amax f- X
12:
else
13:
Amin + X
end
15:
end while
X"
+
Output: Solution Zij for MTBAP with a worst-case approximation ratio of max, Fij OPT
A.2
The Formal Proof of Eq. (11) in Sec.4.2
Proof. Recall that Eq. is given by L (0; DRCT) = -Ei,t NNN.
[.," =ú, (F()), ruz We aim
to show that this is an unbiased estimator of (0) = - BEE(N).JEENN [ (F(O), ((0) - -ry]] Note that LDL (0: Dnct) can be rewritten as:
N
(DI(6; DRCT)
Fiitt
2ie, (f(O), THe
Nu.
N
21, (f(0), c(0)) ra.
N
Nu.
N Zu, (f(0), ra.
Σ Lefe,, (f(0), c(0))
-raltt 5] Σ (F(),, rul (T' (T1X))
M - M | ((()).((0) riy where T' X holds because the data set Dact from random control trials (RCT). Therefore,
N
- -E,,t
No, -.. (F(0).2(0)) Tat, = M - EEELN)
[ (F(0), 2(0)) -rg]
which completes the proof. Note that there a multiplicative factor of M. To ensure consistency, we revise Eq. 3 in the main text (0) -M (EKENN).EMM) [2 (f(0), c(0)) ry].
D
21

---

A.3 More details of The maximum entropy regularization trick in Sec.4.2
The primal policy learning loss LPPL can also be derived through the maximum entropy regularization trick. After obtaining the optimal Lagrange multiplier X via binary search, we introduce a maximum entropy regularizer into the objective function of the dual formulation of the MTBAP:
max ΣΣ
(fis - X Xd))zi - ΣΣ In Zij.
s.t. Σ Zij Vi,
Zy E [0, 1],
where r >> O is the temperature hyperparameter controlling the entropy regularization strength. Note that the dual formulation of MTBAP can be equivalently relaxed E [0. 1]. Further we have:
((2,8)= = (+t Fy X X(xz))
Zi In
2ij
Σ
B
1 1-
Σ 2ij
(19)
@L(2.8)
where 33 represents the dual variables associated with the equality constraints. Setting
0s
and 0L(x.)) yields the optimal solution: ag
exp (fig - X [z)/r]] Σn expl(fix X+T)
(20)
Substituting this into Eq 11 we derive LPPL by the maximum entropy regularization trick:
N
expl(fnr, (0) Cr, (0))/T]
LPPL DRCT) -Bi,
Pite
(21)
Nr.
EjE( expl (fi) (0) (0);
A.4 More details of The Primal Improved Finite Difference Strategy (PIFD) in Sec.4.2
The primal improved finite difference strategy (PIFD) estimates gradients (CDL(0:PReT) via black- (F(),,())
box perturbations LDL. Using the finite difference strategy, the gradient of LDL DRCT) with respect to Fig is estimated as:
@LDL(n,AA.)) Cpi(r.c, + e,h, é) (Dl(r,) Of
where h a small constant, and Eiy E (0, 1xM a matrix where only the element the i-th row and j-th column is 1, and all other elements are 0. The gradient term @Cor. can be computed similarly. We accelerate above computation with the Lrrı--awrre gradient estimator. This involves two key improvements: first, replacing the black-box perturbation with a semi-black-box one; and second, unifying the separate perturbations on and e into single perturbation on z. Together, these changes improve the stability of the gradient and significantly accelerate the solution process. Given the gradients 8Cpt (A.PRCT) and freeze them, this final surrogate decision loss LPIeD is defined as:
(F0)
0Lmr DRCT) ((()) (F() a(0))
LIID((; DRCT) BLE[MNJE(M
@LDL(O; DRCT) 02 (f(() c(0)) ΣyeM exple X+ (0)
expl (0) X
The pseudocode for the Laprr-aware gradient estimator in PIFD is provided in Algorithm 3 For each sample, we first compute the minimal perturbation that alters the primal decision loss, and then update the loss by correcting only the original result. For clarity, Algorithm 3s presented using for loops; in practice, we implement it with matrix operations in order to accelerates computation.
22

---

Algorithm 3 Lppr.-aware gradient estimator of the primal improved finite difference strategy (PIFD)
Input: budget B; Training data D) { li; Fit,; Cit,) ))==1; predicted revenue/cost è.
Compute: For each t in [M], N₂ - number of samples with E, = t; Pt + NI/N.
Initialize: (6m..0:DRecT) 0zly (2()(((0),(()))
2 for all i,j.
Given f, è, and D, call Algorithm to obtain X and Zij-
Ji Gij = (raj X Cit), , 2ij yarg max, (Foz -rs)))) 2: F(A",r, (, < N Ei Pt; Ft, marg max, Fey.
(B, F(A,naa
3: matching indices {i t, arg max, Zij, Vi)
4: mismatching indices = {i t; # arg max, 2ij. Vi}
5: for all E matching_inee ind__iindccssssss do
6: hr = max Oij Cst.
8-LpL N... Tie
7:
h
8:
for all {1,2, M), j # do
9:
h = ait. - -Oij
10: B-LpL N-P4, 1889
his
11: end for
12: end for
13: for all E mismatching indices do
14:
arg.ax, max, aij
15:
hig = =aj Ojt, hi
8-LpL
N.PL,
16:
Nigg
a 8-LpL
NiPe, T94 KE
17:
18:
end for
Output: the gradients
@Col (8:Dec)) (P()),00))
CoL (6:Dner), the optimal Lagrange multiplier A
Jely ((0))
A.5 The Explicit Differentiation Algorithm for Bi-level Optimization Sec.4.3
We introduce the explicit differentiation algorithm for Bi-level Optimization. As discussed in Sec. 4.3, computing the Jacobian 06" (d) in the absence a closed-form solution for (e)) is well-known challenge in bilevel optimization (BLO). A common approach to explicitly differentiate through the gradient descent step. under the assumption that (x)) can be reached in single gradient descent (GD) step 15 [11, as shown below:
te
(*) - - 0gg VeLpI. (6, 0; DOBS).
(22)
By retaining the above update path within any automatic differentiation library, we can explicitly dif- ferentiate through the gradient step to compute gradients with respect to the bridge model parameters o. Specifically, V DDL (#)) DRCT) can be computed as:
00 0
V LDLDL (8* (ner)) = VoLoL DRCT))
VoLDL (0; PRCT)
(V & (-ag VoLeL (g, DoBs)))
(23)
-ag VaLDI PRCT) V VeLpL (0, 0; Dons)
Here, (0) is computed by differentiating through the single gradient descent update in Eq. 22
0
LPL (ò. 0; DoBS)
-00
(24)
dΦ
8000
This approach, known as the Explicit Differentiation Algorithm, enables end-to-end optimization of the bridge model parameters using standard automatic differentiation frameworks. However,
23

---

& Initialize solution the Explicit Differentiation Algorithm relies heavily on the optimization path and, when combined with decision loss, is often susceptible to vanishing gradients and suboptimal solutions. It should be emphasized that the assumption of reaching the optimum in one gradient descent step is often unrealistic. practice, single update typically leads suboptimal solution, whereas multiple updates can result in severe vanishing gradient issues.
A.6 More details of the conjugate gradient (CG) Algorithm in Sec.4.3
In this appendix, we provide additional details on the conjugate gradient (CG) algorithm, which serves as a component within Algorithm I
described in Sec. 4.3. Note that the conjugate gradient (CG) algorithm is employed to efficiently solve the following large-scale linear system:
8 Co 0; DoBS)
(0) (LL 0; DOBS)
002
0Φ00
(=
which is the same as Eq. (8) The core idea the CG algorithm that solving Ar = b equivalent to minimizing the quadratic function 5² Ax-b" x. Moreover, the CG algorithm can be implemented without explicit storage of large matrices by relying solely on matrix-vector products. For example, for Hessian matrix A V2C, the matrix--vecto product Ap be computed as: Ap = Vo (p" VoC), where p is an arbitrary vector. This trick, which uses automatic differentiation twice, also applies to other matrices, enabling efficient implicit computation without explicit matrix construction. The pseudocode for the standard conjugate gradient algorithm summarized in Algorithm
Algorithm 4 The Conjugate Gradient (CG) Algorithm
Input: Matrix A; Vector b; Initial guess zo; Tolerance Maximum iterations Neg-
1: I < Top
2: x- b- Ar
Compute initial residual D Set initial search direction
3: p prr
4: for k -0.1,2..... Peg do
5:
if |r|| < then
6:
break
Converged
7:
end if
TTr
8:
a p Ap Ap
P> Step size
9:
I x+ ap
Update solution
10:
Tnew r a Ap
Update residual
11:
if Nnewl e then
12:
break
D. Converged
end if
14:
B Tace Pare
> Update coefficient
15:
p
Fnew 3p
Update search direction Prepare for next iteration
16:
r
< Fnew
17:
end for
Output: Solution x such that Ax Rd b
A.7 Dual Decision Loss (LDDL.) Dual Policy Learning Loss (CDPr.), and Dual Improved Finite Difference Strategy (DIFD)
We provide an alternative formulation of the decision loss, termed the dual decision loss, which is designed to directly quantify the decision quality of the dual formulations of MTBAP.
(D) - EEEINT JE[M]
Σ
2
dual (f(0), ((0)) (rij X- cxy)
(25)
LDDL is also non-computed due to the lack of the counterfactual responses. By leveraging strong ignorability (X, R(t), C(t)) 1 T of experimental data, we derive unbiased estimator of the dual
decision loss as follows:
N
dual (f(0),2(0)) (ra. A A.c.. 2
CDDL (f: DRCT) -E Σ
(26)
N₂.
24

---

Nr. is the number of individuals assigned treatment y in DRCT. Note that in this context, À is not the optimal Lagrange multiplier X obtained by binary search, but rather a user-specified hyperparameter. It represents a discrete interpolation over arbitrary budget constraints B.
LDDL is continuously differentiable with respect to 2/,, (F(O), c(0). Based the Lagrangian relaxation algorithm A (.. and given arbitrary A, the solution is:
dual Sis, (f(O), (()) 1 { arg max (R) Neiy (00( JELM
(27)
where is indicator function and À is an arbitrary user-specified Lagrange multiplier. Due to the existence of indicator functions, 22, (f(O). a()) is non-differentiable with respect 6. By
utilizing Softmax functions, the discrete solution zdual (f(O), ε(0)) can be relaxed a continuously
differentiable function Sit,
duall (f(0), c(0), which also be regarded the probability of .dual 1: 2it,
explfic, (0) Xdea (0)
dual' (f(0), ((0)) = ΣjE(n explf:; (0) (c:)(0)1
(28)
Hence, we obtain surrogate decision loss CDPL of LDDL, called the dual policy learning loss:
N explfu, (0) Adse, (8)
LDPL (RCT) = Bii,, Σ
(rax, À cies))
(29)
N ΣjE[M explfig (0) xa1z(0)
While the dual loss considers all budget levels, our proposed LPPL in the main text directly targets decision quality under a specific budget B, thereby better aligning with real-world marketing con- straints. We also introduce the Dual Improved Finite Difference (DIFD) strategy, which estimates the gradients Bedual (F())
(8:nec)) via black-box perturbations on LDIL. and accelerates computation using
LDPr.-awan gradient estimator. Compared to LDPL. DIFD preserves the dual optimization landscape without relaxation, and, by freezing the computed gradients as non- trainable nodes, enables seamless integration with automatic diffcrentiation libraries. The surrogate decision loss LDIFD is given by:
dLDDL (0; PRCT), dual' (F(O),
LDIDD DRCT) = E.E[NT)JE[MM
Σ
Daduall (f(0), 2(0)
2ij
(30)
Also, the pseudocode for the CDPL-aware gradient estimator in DIFD is provided in Algorithm
B More Details of Offline Experiment
Here, we provide additional details for Sec.5.1 (Offline Experimental Sctup) and Sec.5.2 (Offline Experimental Results) of the main text.
B.1 Details of Offline Experimental Setup
Here, we provide additional information for Sec. 5.1 (Offline Experimental Setup) of the main text, covering the dataset and preprocessing, evaluation metrics, and experimental details.
B.1.1 CRITEO-UPLIFT v2 (Hybrid)
CRITEO-UPLIFT v2. This public dataset is provided by the AdTech company Critco in the AdKDD'IE 18 workshop[II] The dataset contains 13.9 million samples collected from random control trial (RCT) that prevents a random part of users from being targeted by advertising. Each sample has 12 features, 1 binary treatment indicator and 2 response labels(visid/conversion)) In order to study resource allocation problem under limited budget using the dataset, we follow [51] and take the visit/conversion label the cost/value respectively. To better reflect real-world marketing scenarios where OBS data far outnumbers RCT data, we simulate a marketing policy to convert part of RCT data into OBS data. We refer this as CRITEO-UPLIFT v2 (Hybrid).
CRITEO-UPLIFT v2 (Hybrid). Given total of 13.9 million RCT samples, we use 5% of the data to train two-stage model with the standard cross-entropy loss. This trained model is then used to
25

---

Algorithm 5 Lnpr.-aware gradient estimator of the dual improved finite difference strategy (DIFD)
Input: Lagrange multiplier X; data D ((r,, 4, Fit, P ) == predicted revenue/cost F;
Compute: For each t in [M]] N₁ - number of samples with E, = t; Pt + Nr/N.
Initialize: aCnr, 0= (f(0),a(0))
0, 2ij 0 for all i.j.
1: 3, dej = (Tij - x. Gij), 2ij j= (aa, (xc5))
2: F(A,T, (, - N 2i pt. rr, tar maxy 2zj
3: d(,,r,c,f,)) + v E; pt, Cr.
fr.Firargmax aaaaargaan max, Ray
4: -LDpL(A, e, (. < F(A.r,
r,
5: matching_img
baahhingiindces-tt indices {i t; arg max; Zij. fit
6: mismatching indices {i t, F arg max, Zij. Vi)
7: for all matching _idiicesnceeseseee do
8:
hig = ma (ij - ait.
a B-LDDL -N.p4 (Fr,, x-cve.)
9:
Ozdual
hig
10:
for all j 21.2,... ...; M), j + to do
11:
h = ait - aij
p.; (re;; -recin)
12:
8-LDDL Ozdual
13:
end for end for
14:
15: for
all
mismatching indices do = arg max, aij
16:
17:
h asj Oct, hij
h
d-EpDL
N.pp.
(ri,, -
O=dual
hic. -rre,- -
B-Epei Oedual
Nopt
19:
Kig
20:
end for
Output: the gradients
@Com (0:Dar) aedy (H(0),8(0))
Coo ( Dadinl (f()).2(8))
simulate marketing policy on 50% of the total RCT samples. We construct the observational (OBS) dataset by selecting users for whom the coupon assignment under the simulated policy matches the actual assignment the data. Note that this procedure discards unmatched RCT samples, resulting in an OBS dataset with 3,498,294 samples, which accounts for approximately 25% of the total data. Our analysis shows that the constructed OBS dataset achieves an 82.43% improvement in ROI compared to the random dataset, which demonstrates that the constructed OBS dataset closely reflects observational data generated by real-world marketing strategies. Excluding the 55% of random data that is not utilized in the above process, we further split the remaining 45% RCT samples into 5% for the RCT training set, 10% for validation set, and 30% for test set. To summarize, the resulting datasets contain 3,498,294 samples in the OBS training set, 698,960 in the RCT training set, 1,397,959 in the RCT validation set, and 4, 1918,,878 in the RCT test set. worth noting that the ratio of OBS to RCT samples in the training set is approximately 5:1.
B.1.2 EOM (Expected Outcome Metric).
EOM (Expected Outcome Metric). EOM a common metrie for marketing optimization in 2 49 52.
Based on RCT data, an unbiased estimation of the expected outcome (per-capita revenue/per- capita cost) for arbitrary budget allocation policy can be obtained. Since EOM represents the decision quality of marketing under multilple treatments, we use EOM to compare the performance of different methods in Marketing data I and We give pseudocode of EOM (Expected Outcome Metric) for unbiased estimation of per-capita revenue cost in Algorithm 6
26

---

Unbiased estimation of expected outcome (per-capita revenue or cost) for Lagrangian budget allocation policy A with predicted revenue f and cost é under budget B.
Algorithm 6 EOM:
Input:data D = {(Tr, f4, Fie,, Cit. Di budget B: predicted revenue/cost f, é; small constant Compute: For each in [M],, N₂ < number samples with t, Pt + N//N.
Initialize: Amin f- 0, Amax f- max.,j
24j t Oes
Psi
0 for all i,j
1: while Amax Amin > e do
2: A Amax
3: for all i.j do
4:
Zij + -IG arg max, (frg Acij))
5: end for
6:
F(A.r,
N E: rt (t, = arg max, 2ij)
7: aArrF,, N E, a, (t, arg max, 2ij)
8:
if ad, r. (. f. < then
9: break
10:
end
11:
if
B N
c(), v, n,r,,,f,, > 0 then
12:
Amax t
13:
else
14:
Amin + A
15: end if
16: end while
17: x
18: FB
(B, a(X",
Output: expected per capita revenue F, C), expected per capita cost e(Brr,, (,,
B.1.3 Experimental Details
Model Architecture. For CRITEO-UPLIFT v2 (Hybrid), we employ a 4-layer multi-head multilayer perceptron (MLP) with layer sizes of 64-32-32-4, where the first two outputs correspond predicted revenue and the remaining outputs correspond to predicted cost. For Marketing Data I, we use a layer multi-head MLP with layer sizes of 128 28-66432-118 16; in this case, the first eight outputs represent predicted revenue, and the remaining outputs represent predicted cost. For Marketing Data II, the model is 4-layer multi-head MLP with layer sizes of 128-64-32-10, where the first five outputs for predicted revenue and the remaining outputs are for predicted cost. Note that, unless otherwise specified, the target model, bridge model, and teacher model all adopt the same architecture.
Device. All experiments are conducted on two NVIDIA A100 GPUs with a total of 232 GB memory.
Optimizer. We use the Adam optimizer for training.
Training Procedure. In the three experiments, the models trained for 100, 500, and 500 epochs, respectively. For each experiment, the model checkpoint with the highest AUCC/EOM on the validation set selected as the best model.
Other Hyperparameters. The number of gradient descent (GD) steps for assumed updates, k, set to 5. The number of conjugate gradient iterations, Negs is set to 50. The warm-start period for Bi-DFCL, applicable, is set to 20 epochs.
B.2 Details of Ablation Studies.
To show the effects of individual components, conduct ablation study by incrementally adding four key components Bi-DFCL to baseline a sequential manner: Decision Loss (PPL), Bi-level Optimization by hybrid RCT and OBS data, Counterfactual Labels, and Implicit Differentiation Algorithm. The experimental results on marketing datasets are reported in Table Table
Specifically, the baselines corresponding to each row in Table are described as follows:
27

---

Table 5: Ablation study of each individual component in Bi-DFCL with two marketing datasets.
Compesents Bi-DFCL
Marketing Data
Marketing
Decision ess (P91.) Hi-level Optimization Counterfactaal Labels
Implicis Differentation
BOM
Improvement
ROM
Improvmentt
1,0000
L.0000
1.0167
167%
1.0156
1.56%
X
1,0240
2.40%
1.0175
1.75%
1.0248
2.48%
1.0213
2.13%
1/0277
2.77%
1.0252
2.52%
Row (Baseline): This is the TSM-SL baseline trained on RCT data only, without any of the proposed components. It serves as the basic reference model.
Row 2 (Baseline + Decision Loss): This variant corresponds to DFCL-PPL, which incorporates the decision loss (Cppr.) on RCT data, but does not include bi-level optimization.
Row 3 (Baseline + Decision Loss + Bi-level Optimization): This setting corresponds to Bi-DFCI.- PPI. without using synthesized counterfactual pseudo-labels to parameterize CrI. Instead, it employs an improved version of IPW (inverse propensity weighting), where the bridge model directly outputs dynamically adaptive weights for reweighting factual samples, rather than using fixed or estimated propensity scores. Implicit differentiation is not employed here(i.e., explicit differentiation is used),
This config- uration is Bi-DFCL-PPL, where synthesized counterfactual pseudo-labels are used to parameterize the LPL. but implicit differentiation is still not applied ex.e..explicit differentiation used).
Row 4 (Baseline + Decision Loss Bi-level Optimization + Counterfactual Labels):
Row 5 (Full Model): This is the complete Bi-DFCL with all four components enabled: decision loss (PPL), bi-level optimization, counterfactual labels, and implicit differentiation.
We can find that after the introduction of each module, the performance can all be strengthened to some extent, which demonstrates that our three contributions can all benefit the marketing optimization.
B.3 Details of In-depth Analysis
The effect of RCT and OBS training data size. We first conduct an in-depth analysis to investigate the effect of training data size performance using Marketing Data I, as well to validate the bias-variance properties of RCT and OBS data. The experimental results are summarized in Table 6
Table 6: Effect of training data size (OBS and RCT) on performance with Marketing Data
Method
OBS
RCT
OBS:RCT Ratio
EOM
Improvement
TSM-SL
2,220,781
0
0.9869
-1.31%
TSM-SI.
0)
2,220,781
1.0000
TSM-SI.
22,201,405
()
1.0067
0.67%
Bi-DFCL-PPL.
22,201,405
222,000
100.01:1
1.0190
1.90%
Bi-DFCL-PPL 22,201,405
1,100,000
20.18:1
1.0258
2.58%
Bi-DFCL-PPI.
22,201,405
2,220,781
10.00:1
1.0277
2.77%
As shown in Table g models trained solely on RCT data (e.g., TSM-SL with 2,220,781 RCT samples) serve as an unbiased reference, but their performance is limited by high variance due to the relatively small sample size. In contrast, models trained only on large-scale OBS data may suffer from bias, as reflected in lower EOM values when using 2,017,451 or even 3,381.5274 OBS samples alone. Notably, as the amount of OBS data increases from 2,220,781 to 22,201,,00 the EOM improves from 0.9869 to 1 1.0067, which highlights that the low-variance property of large-scale observational data is highly beneficial for robust and high-quality decision making. Furthermore, when a sufficient amount of RCT data is combined with abundant OBS data (c.g., Bi-DFCL-PPLL with 22,201,405 OBS and 2,220,781 RCT samples), the model achieves the best performance (EOM= 1.0277, Improvement 2.77%). This demonstrates the effectiveness of leveraging large-scale observational data to reduce variance, together with a moderate amount of randomized data to correct for bias, thereby achieving favorable bias- variance trade-off and superior overall model performance.
The sensitivity of key hyperparameters. We further evaluate the sensitivity of key hyperparameters, specifically the number of gradient descent (GD) steps for assumed updates (k, default = 5) and the number of conjugate gradient iterations (Nag. default 50), by varying their values. The results on Marketing Data II are summarized in T
28

---

Table Sensitivity analysis of key hyperparameters on performance with Marketing Data II.
& (GD Steps)
Meg (CG Iterations)
BOM
Improvement
10
1.0199
1.99%
50
1.0217
2.17%
5
10
1.0230
2.30%
5
50
1.0252
2.52%
5
100
1,0253
2.53%
5 200
1,0249
2.49%
10
50
1.0255
2.55%
10
100
1.0252
2.52%
As shown Table the performance of our method is relatively stable across range of values for & and Reg. indicating that the proposed approach is robust to these hyperparameter settings. Notably, when h = 1, the implicit differentiation algorithm does not provide a significant advantage explicit differentiation. This suggests that the strength of implicit differentiation lies in its independence from the optimization path, allowing for any number of iterative updates to reach the optimal solution, rather than relying the overly strong assumption of explicit differentiation that single gradient descent step suffices to achieve optimality.
The Robustness of Bi-DFCI. Moreover, we evaluate the robustness of Bi-DFCL. under multiple sets of budget values B. The results on Marketing Data and are summarized Figure]
1.300
1.175
1.175
ISM-SLIBCT)
ISM-SLIBCD
Increaccnnal Reward (ormmatedd)
1.125
2.100
1.100
Lacretaconaal
1.075
1.000
1.055
1.00
Budget (addes)
(a) Incremental reward (normalized EOM) on Market-
(b) Incremental reward (normalized EOM) on Market- ing Data 11 across 10 budget levels.
ing Data across 10 budget levels.
Figure 3: Robustness of Bi-DFCL under multiple budget values B Marketing Data and
As illustrated in Figure 3 Bi-DFCL consistently achieves higher incremental reward (EOM) across a range of candidate budget values on both Marketing Data and II. This demonstrates the robustness and effectiveness of Bi-DFCL maintaining superior decision quality when the budget varies within several candidate levels, further highlighting its practical applicability in real-world marketing.
Computational Efficiency Analysis. Finally, we provide a comprehensive analysis of the computa- tional overhead of Bi-DFCL from both space and time efficiency perspectives.
Space Efficiency: Bi-DFCL. does not incur additional space overhead compared to existing baselines. While implicit differentiation algorithms typically require storing large-scale inverse matrices, we employ the Conjugate Gradient (CG) algorithm avoid this issue. The CG algorithm circumvents the storage of large-scale inverse matrices through matrix-vector products (see Appendix A.6).
For online inference, Bi-DFCL only uses the well-trained target model, resulting in inference time identical to simple causal learning methods. However, additional time overhead occurs during offline training. Table compares the training time of different methods on Marketing Data II.
Time Efficiency:
For fairness, all methods were fully trained for 500 epochs using the same model structure (no early stopping). As shown in Table Bi-DFCL requires approximately 6-7 times the training time of the simplest causal method TSM-SL. The ablation studies reveal that most time overhead stems from solving the bi-level optimization problem. Our use of implicit differentiation with the CG algorithm

---

Table 8: Comprehensive analysis of training time (minutes) across different methods
Method
Data
Training Time (min)
Relative to TSM-SL
TSM-SL
RCT
2.505
0.06x
DFCL-PPL
RCT
3.163
0.07x
DFCL-PIRD
RCT
9.948
0.23x
TSM-SL
OBS
39.918
0.94x
TSM-SL
RCT+OBS
42.332
1.00x
KD-Label
RCT+OBS
67.358
1,59x
LID-DR
RCT+OBS
492.559
11.63x
AutoDebias
RCT+OBS
397.886
9.40x
Bi-DFCI--PPL
RCT+OBS
265.263
6.26x
Bi-DFCL-PIID
RCT+OBS
294,927
6,96x
Bi-DFCL-PPEL w/o
RCT+OBS
345.132
8.15x
Bi-DFCL-PIFD w/o ID
RCT+OBS
427.515
10.10x
provides two key advantages: (1) it reduces time complexity from O(n") to O(n) by avoiding matrix inversion, and (2) obtains more accurate optimal solutions, allowing bilevel optimization solving once every & batches rather than every batch. The comparison between Bi-DFCL variants with and without implicit differentiation (ID) demonstrates the efficiency gains of our approach. In summary. although Bi-DFCL introduces additional offline training time, this investment is justified by significantly improved online decision quality. Our further improvements also effectively mitigate this overhead, making Bi-DFCL practical for real-world marketing applications.
C Boarder Impacts
Our work offers several positive societal impacts. First, by improving the decision quality of marketing resource allocation, method helps platforms maximize the effectiveness of their marketing campaigns under real-world budget constraints. This can lead to increased user engagement and satisfaction, users more likely receive relevant and timely offers. Second, the reduction of resource waste contributes to more sustainable business operations, which benefits both companies and consumers. Third, our approach has demonstrated strong performance in both offline benchmarks and large-scale online deployments, indicating its practical value for the digital economy. The adoption of such data-driven decision-making tools further support innovation and the healthy development of the broader digital marketing ecosystem.
30


Reviews:
[{"review_id": "Q8ornCQJvx", "reviewer": "Reviewer_rQbD", "summary": "The paper proposes Bi-Level Decision-Focused Causal Learning (Bi-DFCL), a framework for large-scale marketing optimization that addresses prediction-decision misalignment and bias-variance tradeoffs by leveraging both observational and experimental data. It introduces an unbiased estimator of decision quality and a bi-level optimization framework to dynamically correct learning directions, achieving superior performance in offline and online tests.", "questions": "- While Bi-DFCL demonstrates superior performance, could the authors elaborate on the computational overhead of the bi-level optimization compared to simpler causal methods? Real-time deployment constraints in marketing systems are critical.\n\n- Can authors discuss further on how would it perform in domains with no experimental data (e.g., healthcare), given the paper’s focus on marketing?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n- The idea of addressing prediction-decision misalignment and bias-variance tradeoffs is novel.\n\n- This method has been employed in real-world application and has demonstrates=d significant revenue improvements (e.g., 3.5% lift in orders), proving scalability and practicality.\n\nWeakness:\n - The generalization scope could be a weakness. This method highly relies on hybrid RCT/OBS data, which might restrict applicability to platforms where experimental data is extremely scarce.\n\n- Some key components of this paper, e.g., sensitivity of the hyper-parameters are expected to appear in the main text. However, authors put them into appendix.", "was_revised": true, "final_justification": "This authors have solved my concerns in the time complexity and potential applications in scenarios where experimental data is less available. Thus I will keep my evaluation that this paper should be accepted.", "rebuttal": "We sincerely appreciate your approval of the idea and the novelty of this work, and thank you for the helpful suggestions. Below, we hope to address your concerns and questions to improve the clarity and readability of our paper.\n\n---\n\n### **【Q1】The  computational overhead**\n\n**[Q1]: ”While Bi-DFCL demonstrates superior performance, could the authors elaborate on the computational overhead of the bi-level optimization compared to simpler causal methods? Real-time deployment constraints in marketing systems are critical.”**\n\n**[Q1Response]:** Thanks for the question. \n\n1. **Space Efficiency**: Details of space efficiency can be find in **Appendix B.1.3 Experimental Details**, Our proposed Bi-DFCL **does not incur additional space overhead** compared to existing baselines. \n\n\t* Note that implicit differentiation algorithm itself requires the storage of large-scale inverse matrices, which have high demands on computing resources.\n\t* To enable the application of implicit differentiation in large-scale marketing optimization, we use the **Conjugate Gradient Algorithm (CG)** to avoid this problem—the conjugate gradient method **avoids the storage of large-scale inverse matrices through the trick of matrix-vector products**(see lines 239-245 in the main text and Appendix A.5 for specific details).\n\n2. **Time Efficiency**: \n\n\t* **Online Inference:**  \n\n\t\t* We first want to clarify that for the online deployment and inference of Bi-DFCL, we only use the well-trained Target Model (see the Output of Algorithm 1). \n\t\t* Therefore, **the inference time of our method is exactly the same as that of simple causal learning methods—because their model structures are the same**. \n\n\t* **Offline Training:**\n\n\t\t* However, Bi-DFCL does introduce additional time overhead during offline training—which is one of the issues we have been working to address. We analyze the time overhead of Bi-DFCL from both experimental and theoretical perspectives.\n\n\t\t1. **Experiments of Time Effciency**\n\n\t\t\t* As suggested by the reviewer, we compare the training time (minutes) of different baselines (using RCT + OBS data for training) on the Marketing data II. The results are shown below.\n\n\t\t\t* | Dataset II | Methods          | Training time |\n\t\t\t\t| :--------: | :--------------- | :-----------: |\n\t\t\t\t|  RCT+OBS   | TSM-SL           |    42.332     |\n\t\t\t\t|  RCT+OBS   | KD-Label         |    67.358     |\n\t\t\t\t|  RCT+OBS   | LTD-DR           |    492.559    |\n\t\t\t\t|  RCT+OBS   | AutoDebias       |    397.886    |\n\t\t\t\t|  RCT+OBS   | **Bi-DFCL-PPL**  |  **265.263**  |\n\t\t\t\t|  RCT+OBS   | **Bi-DFCL-PIFD** |  **294.927**  |\n\n\t\t\t* Note: For fairness, all methods were fully trained for 500 epochs using the same model structure (no early stopping).\n\n\t\t\t* From the table, the following conclusions can be drawn: \n\n\t\t\t\t* As one of the simplest causal learning method, TSM-SL has a training time of approximately 42 minutes.\n\t\t\t\t* The training time of Bi-DFCL is about **6～7 times that of TSM-SL**.\n\n\t\t2. **Theoretical Analysis about Time Effciency**\n\n\t\t\t* We decompose the additional time overhead of Bi-DFCL compared to TSM-SL into two parts: \n\n\t\t\t\t1. The time overhead of the Decision Loss; \n\t\t\t\t2. The time overhead of solving the bi-level optimization problem;\n\n\t\t\t*  We supplement the following ablation experiments: \n\n\t\t\t* | Dataset II |        Methods         | Training time |\n\t\t\t\t| :--------: | :--------------------: | :-----------: |\n\t\t\t\t|    RCT     |         TSM-SL         |     2.505     |\n\t\t\t\t|    RCT     |        DFCL-PPL        |     3.163     |\n\t\t\t\t|    RCT     |       DFCL-PIFD        |     9.948     |\n\t\t\t\t|    OBS     |         TSM-SL         |    39.918     |\n\t\t\t\t|  RCT+OBS   |         TSM-SL         |    42.332     |\n\t\t\t\t|  RCT+OBS   |    **Bi-DFCL-PPL**     |  **265.263**  |\n\t\t\t\t|  RCT+OBS   |    **Bi-DFCL-PIFD**    |  **294.927**  |\n\t\t\t\t|  RCT+OBS   | Bi-DFCL-PPL-wo ID Alg  |    345.132    |\n\t\t\t\t|  RCT+OBS   | Bi-DFCL-PIFD-wo ID Alg |    427.515    |\n\n\t\t\t* It can be seen that most of the time overhead of Bi-DFCL comes from solving the bi-level optimization problem. \n\n\t\t\t* To accelerate this process, we use the implicit differentiation algorithm and CG algorithm for solving, and it can be observed that **one of the advantages of implicit differentiation is lower time overhead**. \n\n\t\t\t* \"**Bi-DFCL-wo ID Alg**\" indicates that the solution to the bi-level optimization problem no longer uses the implicit differentiation algorithm, but instead uses the more common explicit differentiation algorithm (see Appendix A.4 for specific details).\n\n\t\t\t* **What makes the implicit differentiation algorithm faster?**: \n\n\t\t\t\t* The CG algorithm efficiently solves the following large-scale linear system through numerical methods, thereby avoiding matrix inversion — reducing the time complexity from O(n³) to O(n).\n\t\t\t\t* Implicit differentiation, as it does not depend on the optimization path, can obtain a more accurate optimal solution for the lower-level optimization problem compared to explicit differentiation(assuming one step GD to reach optimal). This allows solving the bi-level optimization problem once every k batches, rather than having k=1 as in explicit differentiation.\n\n\t\t3. **Summary**\n\n\t\t\t* Bi-DFCL does not incur additional space overhead or inference time overhead, but it **does have a training time overhead 6-7 times that of the baseline TSM-SL**. \n\t\t\t* However, on the one hand, the extra  **offline training time is absolutely worthwhile in exchange for higher online decision quality**. On the other hand, we have **also made several improvements to address this issue (such as the CG algorithm)**.\n\n\n\n---\n\n### **【Q2】Domains with no experimental data**\n\n**[Q2]: ”Can authors discuss further on how would it perform in domains with no experimental data (e.g., healthcare), given the paper’s focus on marketing?”**\n\n**[Q2Response]:** \n\n* We acknowledge that there are some causal inference scenarios where no RCT data is available. However, it is regrettable that this paper assumes the domain of marketing optimization (where a small amount of RCT data can be obtained). \n* Meanwhile, we believe that Bi-DFCL has certain value in other scenarios beyond marketing optimization where RCT data can be obtained even when extremely scarce. We conducted experiments in **Appendix B.3 (Details of In-depth Analysis), Table 6**, which demonstrate that our method still has high application value when RCT data is extremely sparse (e.g., only 1%).\n* However, scenarios without any available RCT data are beyond the scope of this paper, but this will be our future research direction. We apologize for this and hope the reviewers will understand. We greatly appreciate the pointing out of this issue.\n\n---\n\n**【W2】: ”Some key components of this paper, e.g., sensitivity of the hyper-parameters are expected to appear in the main text. However, authors put them into appendix.”**\n\n**[W2Response]:** We appreciate the reviewer pointing this out. Due to page limitations during submission, some important content had to be placed in the appendix. In the subsequent revision, we will definitely move the important content from the appendix to the main text. Thank you again for the reviewer's suggestion.\n\n---\n\n**We sincerely thank you for your feedback and will provide more clarifications and explanations in the revised version, and welcome any further technical advice or questions on this work and we will make our best to address your concerns.**"}, {"review_id": "sJIPygr1Fk", "reviewer": "Reviewer_pm1M", "summary": "The paper proposes a Bi-Level Decision-Focused Causal Learning (Bi-DFCL) framework for large-scale marketing optimization. It addresses two key challenges in existing approaches: prediction decision misalignment and bias-variance dilemma. The paper assumes availability of both RCT and observational data and combines tasks of: a) prediction and b) decision to recommend optimal allocation under constraints. The paper performed extensive experiments on a diverse set of data sources to support their claims.", "questions": "The main questions I had were: \n1. Ln180: why would L_{PL} benefit from training only on observational data? Wouldn't the prediction loss also benefit from being trained on RCT data (if available)? That way the quality (in terms of accuracy) of counterfactual allocations can be measured correctly. \n2. Given RCT data + OBS data, is it not possible to train an ITE estimator (possibly for multi-treatment) and use it for L_{DL}? Is there any advantage of the proposed approach over this simplistic approach? \n3. The paper does not discuss the amount of RCT data available vs amount of OBS data. A practical assumption would be that RCT data is much smaller in size as compared to observational data. Also, the experimental setup (Table1) seems to have similar orders of RCT and observational training data. Is there any theoretical or experimental results which shows the performance where the RCT data is orders of magnitude smaller than the observational data (which is the expected setup in practice)?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths: \n1. The paper discusses an important problem for marketing optimisations. \n2. The paper is well written and the approach is clearly presented. \n\nWeaknesses: \n1. The paper does not discuss why the prediction loss should be trained only on observational data. In the presence of observational + RCT data, it is much more efficient to train ITE estimators and use them for downstream optimisation. \n2. The experimental setup (Table1) uses a similar amount of RCT and observational data which seems impractical in most scenarios. \n3. Claims in Section 4.1 does not seem to be correct. If there is ample amount of RCT data available, then both L_{PL} and L_{DL} would benefit from it i.e., RCT data does not selectively improve only one of these losses.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate the reviewer’s great efforts and comments.  Below, we hope to address your concerns and questions to improve the clarity and readability of our paper.\n\n---\n\n\n\n### **【Q3】The amount of RCT data vs OBS data**\n\n**[Q3.1]: “The paper does not discuss the amount of RCT data vs OBS data”**\n\n**[Q3.2]: “The Table1 seems to have similar orders of RCT and OBS training data”**\n\n**[Q3.3]: “The performance where the RCT data is orders of magnitude smaller than OBS data”**\n\n**[Q3Response]**: We apologize for the misunderstanding caused.\n\n1. **We have discussed the RCT-OBS ratios from 1:10 to 1:100.**\n\t* We clarify that the amount between RCT and OBS data **have been discussed** in both the main text and Appendix, with various ratios (**1:10–>1:100**).\n\n\t* In **Table 1 of the main text** , we present three datasets with distinct RCT:OBS training data ratios :\n\t\t* CRITEO: **~1:5** (698,980 RCT vs. 3,498,294 OBS)  \n\t\t* Marketing Data I: **~1:10** (2,220,781 RCT vs. 22,201,405 OBS)   \n\t\t* Marketing Data II: **~1:17** (2,017,450 RCT vs. 33,815,274 OBS)  \n\n\t* In **Supplementary Material: Appendix B.3 (Details of In-depth Analysis), Table 6** supplements these results with additional ratios:\n\t\t* Marketing Data I: **~1:20** (1,100,000 RCT vs. 22,201,405 OBS)   \n\t\t* Marketing Data I: **~1:100** (222,000 RCT vs. 22,201,405 OBS)   \n\n\t* These all validate Bi-DFCL’s performance under extreme RCT scarcity (ratios of 1:10–>1:100) .\n\n2. **The 1:10~1:100 of RCT-OBS ratios in the real marketing scenarios are common and representative**.\n\t* We believe that the statement that \"the RCT and OBS data in Table 1 are of similar orders\" may be a **misunderstanding**.\n\n\t* **1:10~1:100 of RCT-OBS ratios in our experiments are representative of real marketing scenarios** , which are consistent with standard practices in the large-scale marketing optimization— **RCT data is typically collected from 1~ 10% of platform traffic**. **This results that RCT are 10–100 times smaller than OBS data**.\n\n3. **Evidence: Amount of our RCT data are also consistent with existing methods**.\n\t* Existing SOTA methods for large-scale marketing optimization (e.g. DRP[1], RERUM[2],  KUAI-NES[3], DHCL [4], DFCL [5]) typically rely on **millions of RCT data** for training.\n\t* **Our RCT data volumes (2-3million RCT samples) are consistent with these precedents**.\n\n---\n\n\n\n### **【Q1】$L\\_{PL}$  trained only on OBS data or OBS+RCT data**\n\n**[Q1]: “Why would $L\\_{PL}$ benefit from only on obs data? Wouldn't $L\\_{PL}$ also benefit from being trained on RCT data ? ”**\n\n**[Q1Response]：**\n\n1. **Practical Evidence:  small gains of directly adding RCT data to  $L\\_{PL}$**.\n\n\t*  While $L\\_{PL}$ can use a mix of OBS+RCT data, the incremental benefit of adding RCT data is almost negligible in practice due to the sheer scale of OBS data.\n\n\t*  As shown in Table 2 , TSM-SL exhibit minimal improvements when trained on OBS+RCT compared to OBS-only:\n\n\t\t-  Marketing Data I: TSM-SL (OBS) achieves an EOM of 1.0067, while TSM-SL (OBS+RCT) improves to 1.0071—a little gain of **0.04%**.\n\t\t-  Marketing Data II: TSM-SL (OBS) achieves 0.9957, while TSM-SL (OBS+RCT) reaches 0.9988—a little gain of **0.31%**.\n\n\t*  These results align with our expectation: since OBS data is magnitude larger, RCT data contributes little.\n\n2. **Theoretical Analysis: Do not conflict with our assumptions**\n\n\t* We agree that using mixed data to train $L\\_{PL}$  may be a little helpful, but it does not conflict with Bi-DFCL.\n\n\t* **Our key assumption is that $L\\_{DL}$ must use RCT data,  $L\\_{PL}$ can use any types of data.   We did not state that $L\\_{PL}$ *cannot* use RCT data**.\n\n\t* Because the core difference between RCT and OBS lies in whether X and T are orthogonal. If RCT and OBS are simply mixed, X and T remain non-orthogonal. Therefore, **from a theoretical perspective, RCT + OBS is equivalent to OBS**.\n\t\t* OBS data is inherently biased due to non-random treatment assignment ( X $\\not\\perp$ T \\), while RCT data is unbiased because of randomization (X $ \\perp$ T\\)\n\t\t* However, when combined, the overwhelming volume of OBS data (10–100x RCT data) ensures the mixed dataset retains X $ \\not\\perp$ T overall. Thus, mixed data does not resolve the fundamental bias in OBS data\n\n---\n\n\n\n### **【Q2&&W3】Given RCT+OBS, Train an ITE estimator using $L\\_{DL}$?**\n\n**[Q2.1]: “Given RCT+OBS data, is it not possible to train an ITE estimator using $L\\_{DL}$ ? ”**\n\n**[Q2.2]: “Is there any advantage over this simplistic approach ? ”**\n\n**[W3] : “If there is ample amount of RCT data, then both $L\\_{PL}$  and $L\\_{DL}$ would benefit from it , RCT data does not selectively improve only one. ”**\n\n**[Q2&&W3Response]**：Good question.\n\n1. **[Q2.1Response]: $L\\_{DL}$ must be trained exclusively on RCT data**.\n\n\t1. **Theoretical Analysis**:\n\n\t\t* **Unbiased OR Estimator**:\n\n\t\t\t* $L\\_{DL}$ serves as an unbiased estimator of decision quality, and its derivation critically relies on the unbiased nature of RCT data (as proven in Appendix A.2).\n\n\t\t\t* **If using biased OBS data or OBS+RCT data ( X $ \\not\\perp$ T ) , $L\\_{DL}$  is absolutely a wrong  and useless loss**.\n\n\t\t* **Extremely Sensitive To Bias**:\n\n\t\t\t* As discussed in Section 4.1—“$L\\_{DL}$ is highly dependent on the unbiasedness of RCT: minimizing $L\\_{DL}$ on biased OBS would greatly amplify bias and severely degrade decision quality”.\n\t\t\t* OBS+RCT data still suffers from huge bias, which is almost equivalent to only OBS.\n\n\t2. **Practical Experiment**:\n\n\t\t* Note that when using OBS or OBS+RCT for $L\\_{DL}$  , the performance becomes extremely poor.\n\n\t\t* |  Data   |      Loss      | EOM (dataset I) | EOM (dataset II) |\n\t\t\t| :-----: | :------------: | :-------------: | :--------------: |\n\t\t\t|   RCT   |   $L\\_{PL}$    |     1.0000      |      1.0000      |\n\t\t\t|   RCT   | $L\\_{DL-PIFD}$ |     1.0170      |      1.0153      |\n\t\t\t|   OBS   |   $L\\_{PL}$    |     1.0067      |      0.9957      |\n\t\t\t|   OBS   | $L\\_{DL-PIFD}$ |   **0.9703**    |    **0.9542**    |\n\t\t\t| RCT+OBS |   $L\\_{PL}$    |     1.0071      |      0.9988      |\n\t\t\t| RCT+OBS | $L\\_{DL-PIFD}$ |   **0.9727**    |    **0.9561**    |\n\n\t3. **Evidence: Custom loss always use only RCT from existing works. (see Section 2)**\n\n\t\t* Existing studies that address prediction-decision misalignment by **constructing custom losses (e.g., DRP[1], RERUM[2],  KUAI-NES[3], DHCL [4], DFCL [5]) exclusively rely on RCT data**. In their frameworks, no OBS data is leveraged, as these custom losses—similar to our $L\\_{DL}$ —**are inherently dependent on the unbiasedness of RCT to ensure valid decision quality estimation**.\n\n\t\t* In contrast, **methods that do utilize OBS data** (e.g., IPS [6], DragonNet [7]) **are restricted to standard loss (e.g., MSE) for prediction**, which leads to prediction-decision misalignment.\n\n2. **[Q2.2Response]: Advantage of Bi-DFCL over this simplistic approach**.\n\n\t*  As discussed above, **simplistic approach does not work—$L\\_{DL}$ must be trained exclusively on RCT data**. Existing works either use RCT+custom loss or use OBS+standard loss.\n\n\t*  **One key contribution of Bi-DFCL is solving this bias-variance tradeoff**: it effectively incorporates OBS data to reduce variance  (via $L\\_{PL}$ in the lower level) while using $L\\_{DL}$  (must trained on RCT in the upper level) to calibrate biases of OBS data. **This design retains the unbiased decision estimation of  $L\\_{DL}$  while harnessing OBS data for improved generalization—addressing both limitations of existing works**.\n\n3. **[W3Response]: The extent to which RCT data benefits $L\\_{PL}$ and $L\\_{DL}$  differs significantly**.\n\n\t* Yes. We agree that both  $L\\_{PL}$  and  $L\\_{DL}$ can benefit from RCT data when it is abundant.\n\n\t* However, **there is a significant difference in the degree of their dependence on RCT data**:  \n\n\t\t1. **RCT is crucial for  $L\\_{DL}$ ; without RCT data, $L\\_{DL}$ becomes unusable**.\n\n\t\t\t* **Experiments:**\n\t\t\t\t* Marketing data I: 0.9703(OBS+$L\\_{DL}$) —> 0.9727(OBS+RCT+$L\\_{DL}$) —> 1.0170(RCT+$L\\_{DL}$)\n\t\t\t\t* Marketing data II: 0.9542(OBS+$L\\_{DL}$) —> 0.9561(OBS+RCT+$L\\_{DL}$) —> 1.0156(RCT+$L\\_{DL}$)\n\n\t\t\t* **Explaination**:   $L\\_{DL}$ is extremely sensitive to bias of OBS data.\n\t\t\t\t* **$L\\_{DL}$ optimizes the overall optimization problem** rather than the prediction error of distance at the individual sample level. Therefore, relative relationships between different samples interfere with each other.\n\t\t\t\t* Biased data will cause $L\\_{DL}$ to severely distort the directions of the entire optimization problem, **$L\\_{DL}$ becomes a wrong loss when using OBS or OBS+RCT.**\n\t\t\t\t* **So unbiased RCT data(without any OBS data) helps $L\\_{DL}$ very much.**\n\n\t\t2. In contrast, **the benefit of RCT data to  $L\\_{PL}$  is rather limited and can be negligible.** (Of course, we also support your approach of training  $L\\_{PL}$  using a mixture of RCT+OBS data).\n\n\t\t\t* **Experiments:**\n\t\t\t\t* Marketing data I: 1.0067(OBS+$L\\_{PL}$ ) —> 1.0071(OBS&RCT+$L\\_{PL}$ ) —> 1.0000(RCT+$L\\_{PL}$ )\n\t\t\t\t* Marketing data II: 0.9957(OBS+$L\\_{PL}$ ) —> 0.9988(OBS&RCT+$L\\_{PL}$ ) —> 1.0000(RCT+$L\\_{PL}$ )\n\n\t\t\t* **We can see that 1/10～1/20 RCT data helps $L\\_{PL}$ relatively little.**\n\n---\n\n**We hope the above discussion will fully address your concerns, and we would really appreciate it if you could be generous in raising your score.** We look forward to your insightful and constructive responses to further help us. Thank you!\n\n---\n\n### **References**\n\n[1] Improve User Retention with Causal Learning\n\n[2] Rankability-enhanced revenue uplift modeling framework for online marketing\n\n[3] An End-to-End Framework for Marketing Effectiveness Optimization\n\n[4] Direct heterogeneous causal learning for resource allocation problems in marketing\n\n[5] Decision focused causal learning for direct counterfactual marketing optimization\n\n[6] Recommendations as treatments: Debiasing learning and evaluation\n\n[7] Adapting neural networks for the estimation of treatment effects"}, {"review_id": "o9YcxwehHo", "reviewer": "Reviewer_9osN", "summary": "This paper proposes a new method, Bi-level Decision-Focused Causal Learning (Bi-DFCL), for effective marketing resource allocation. The proposed method leverages both observational and experimental data to address challenges in resource allocation problems, i.e., the prediction-decision misalignment and bias-variance dilemma.", "questions": "1)\tI am wondering if the proposed method has been tested on appropriate datasets. The strength of randomized controlled trials (RCTs) lies in random assignment, which ensures that treatment is independent of potential outcomes. The proposed method also relies on the ignorability assumption of the experimental data to derive unbiased estimators. Here, one of my concerns is whether the training, validation, and test sets—derived from each of the datasets (CRITEO-UPLIFT v2, Marketing Data I, and Marketing Data II)—still uphold this assumption. Since the original datasets, which presumably satisfied the ignorability assumption, were divided arbitrarily into three subsets, I am concerned whether the resulting subsamples still retain the characteristics of RCTs. Are their control and treatment groups still balanced after the arbitrary division?\n\n2)\tRelated to the above point, I am also wondering how the real-world A/B tests were designed. Since the proposed method relies on experimental data to derive unbiased estimates, ensuring that the experimental data meet RCT assumptions is critical; Otherwise, the performance results of the proposed method are tenuous. The authors conducted two experiments on the same food delivery platforms; One is about the money-off promotion, and the other is about discounts. Were they conducted independently and during different time periods? Were there any external shocks during the experiment (e.g., holidays, weather) that could have confounded results? Are the 180 and 192 features in each experiment pre-treatment covariates? In real deployments, various factors, other than treatment, can be involved and dramatically bias results. The authors may want to provide further details on the description of the real-world A/B tests. \n\n3)\tI would like to better understand how the proposed method stands up against the existing alternatives. In Table 2, several existing methods (e.g., AutoDebias, LTD-DR) also use both RCT and observational data. The authors may need to clarify how Bi-DFCL compares against these hybrid approaches (not only the ones that rely on either RCT or observational data, as the authors did) and what the pros and cons of these alternative approaches are.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "The idea of utilizing the advantages of different types of data, i.e., experimental data for deriving unbiased estimators and observational data for acquiring more samples and generalizability, is brilliant, and the evaluation setup to verify the proposed method's effectiveness is commendable. However, a few clarifications and improvements are still needed, which are described in the section below.", "was_revised": true, "final_justification": "My comments are mostly about missing, but essential information in the paper. Since the authors clarified all the questions I raised in the rebuttal, I keep my ratings.", "rebuttal": "We sincerely appreciate your approval of the idea and the evaluation setup of this work, and thank you for the helpful suggestions. Below, we hope to address your concerns and questions to improve the clarity and readability of our paper.\n\n---\n\n\n\n**[Q1]: “I am wondering if the proposed method has been tested on appropriate datasets. The strength of randomized controlled trials (RCTs) lies in random assignment, which ensures that treatment is independent of potential outcomes. The proposed method also relies on the ignorability assumption of the experimental data to derive unbiased estimators. Here, one of my concerns is whether the training, validation, and test sets—derived from each of the datasets (CRITEO-UPLIFT v2, Marketing Data I, and Marketing Data II)—still uphold this assumption. Since the original datasets, which presumably satisfied the ignorability assumption, were divided arbitrarily into three subsets, I am concerned whether the resulting subsamples still retain the characteristics of RCTs. Are their control and treatment groups still balanced after the arbitrary division?”**\n\n**[Q1Response]:**  We confirm that all dataset splits maintain RCT properties through three evidence-based guarantees:\n\n1. **Original RCT Collection Mechanism**: Treatments were assigned purely randomly independent of pre-treatment covariates $X$, ensuring: $(Y_i(0),Y_i(1)) \\perp T$ and $X\\perp T$.\n\n2. **Stratified Random Splitting Procedure**: For each dataset (CRITEO-UPLIFT v2, Marketing Data I/II), we performed covariate-agnostic random splitting: Train/validation/test partitions created via simple random sampling. Splitting criteria excluded all $X$ variables, thus still preserving: $(Y_i(0),Y_i(1)) \\perp T$ and $X\\perp T$.\n\n3. **Post-Splitting Balance Verification**: Prior to model training and testing, we assessed covariate balance across treatment groups using Standardized Mean Difference (SMD). Across all datasets, SMD values were consistently below 0.1 (standard balance threshold), confirming balanced covariate distributions between treatment groups. \n\n---\n\n\n\n**[Q2] : “Related to the above point, I am also wondering how the real-world A/B tests were designed. Since the proposed method relies on experimental data to derive unbiased estimates, ensuring that the experimental data meet RCT assumptions is critical; Otherwise, the performance results of the proposed method are tenuous. The authors conducted two experiments on the same food delivery platforms; One is about the money-off promotion, and the other is about discounts. Were they conducted independently and during different time periods? Were there any external shocks during the experiment (e.g., holidays, weather) that could have confounded results? Are the 180 and 192 features in each experiment pre-treatment covariates? In real deployments, various factors, other than treatment, can be involved and dramatically bias results. The authors may want to provide further details on the description of the real-world A/B tests.”**\n\n**[Q2Response]:** Additional Details on Offline Marketing Datasets and Online A/B Experiment:\n\nOn the online food delivery platform, we collected two offline marketing campaign datasets and conducted one online A/B test. These datasets and the online experiment were conducted independently at different times.\n\n1. **Marketing Data I**: Collected during August–September 2024, corresponding to a money-off campaign.\n\n2. **Marketing Data II**: Collected in March 2025, corresponding to a discount campaign. The campaign format difference stems from product strategy adjustments implemented in January 2025.\n\n3. **The online A/B experiment** was conducted in April 2025 using the model trained on **Marketing Data II** and aligned with the discount campaign.\n\nFor both RCT data (from Marketing Data I-II) and the online A/B experiment, some external factors (e.g., holidays, weather, supply-demand fluctuations, or competitor actions) may have existed during the experimental periods. However, the treatment assignments (in offline RCT data) and experiment groups (in online A/B tests) were independent of all covariates—both observed and unobserved. Thus, external factors affected all treatment/experiment groups equally and the balance assumptions for RCTs and A/B tests remain valid.\n\nAll features used in the offline marketing datasets and online A/B tests (180 features for Marketing Data I; 192 for Marketing Data II and online tests) are pre-treatment covariates.\n\n---\n\n\n\n**[Q3] :“I would like to better understand how the proposed method stands up against the existing alternatives. In Table 2, several existing methods (e.g., AutoDebias, LTD-DR) also use both RCT and observational data. The authors may need to clarify how Bi-DFCL compares against these hybrid approaches (not only the ones that rely on either RCT or observational data, as the authors did) and what the pros and cons of these alternative approaches are.”**\n\n**[Q3Response]:** Thanks for the question. \n\n* It is true that some existing methods (e.g.,CausE, KD, AutoDebias, LTD-DR) also propose to utilize RCT and OBS data more effectively, which shares similarities with our idea. \n* However, our method has the following innovations compared to these approaches: \n\t1. **Innovation in addressing prediction-decision misalignment**: \n\t\t* First, one of the biggest differences between our method and these baselines that also utilize RCT+OBS data is that we consider the issue of prediction-decision misalignment. \n\t\t* As stated in the main text, prediction-decision misalignment is a challenging problem in marketing optimization scenarios. However, **existing RCT+OBS baselines only focus on improving prediction accuracy**, which will encounter severe prediction-decision misalignment problems in actual marketing optimization and causal inference scenarios.\n\t\t* We have solved this by deriving two innovative differentiable decision losses— PPL and PIFD, and we have further innovated in addressing the prediction-decision misalignment itself by designing decision losses based on the primal problem.\n\t2. **Innovation in the form of the parameterized prediction loss**: \n\t\t* Second, we have also made innovations in the construction of the parameterized prediction loss. \n\t\t* **Most existing methods rely on reweighting the loss**, such as IPS, AutoDebias, LTD. \n\t\t* In practice, we found that this reweighting method is very unstable in effect — because the weights may be learned to be close to 0 or very large — thus causing excessive changes to the loss and resulting in high variance and instability. \n\t\t* To address this issue, we use a parameterized prediction loss with **counterfactual pseudo-labels**, which are more stable.\n\t\t* In addition, we also incorporate knowledge distillation — instead of directly outputting pseudo-labels, we enable the bridge model to output **gating coefficients**, thereby adaptively weighting the outputs of two different teacher models to combine and obtain counterfactual pseudo-labels. \n\t\t* The pseudo-labels generated by this gating-based method are more stable, mitigating the problem of high variance and the need for warm starts.\n\t3. **Innovation in solving BLO problems—Implicit Differentiation+CG Algorithm**: \n\t\t* Third, we have innovated in solving bi-level optimization problems.  \n\t\t* Existing methods mainly use explicit differentiation, which assumes optimality can be achieved in one gradient descent step (see Appendix A.4) but is path-dependent, prone to vanishing gradients and suboptimal solutions—unrealistic in practice, as single updates yield suboptimal results and multiple updates exacerbate gradient vanishing.  \n\t\t* We instead adopt implicit differentiation, which is path-independent, yielding more accurate and stable gradients.   \n\t\t* While it faces time/space overhead, we address this with the CG algorithm, avoiding large matrix storage and reducing time complexity from O(n³) to O(n). \n\t\t* In summary, **our algorithm for solving bi-level optimization problems can achieve more accurate and stable solutions with higher efficiency**.\n\t4. **Innovation in real-world application scenarios**：\n\t\t* Finally, it is worth emphasizing that existing RCT+OBS baseline studies (e.g., CausE, KD, AutoDebias, LTD-DR) focus on public datasets for recommendation systems, with no methods truly applicable to large-scale industrial scenarios of marketing optimization and causal inference.  \n\t\t* In contrast, our method targets two key challenges in these areas, with its effectiveness validated by multiple offline experiments and online A/B tests. It has been deployed in real scenarios, generating business benefits.\n\n---\n\n**We sincerely thank you for your feedback and will provide more clarifications and explanations in the revised version, and welcome any further technical advice or questions on this work and we will make our best to address your concerns.**"}, {"review_id": "4yDePiy2rZ", "reviewer": "Reviewer_LRH2", "summary": "The paper proposes Bi-Level Decision-Focused Causal Learning (Bi-DFCL), a novel framework for marketing optimization that addresses two key challenges: (1) the misalignment between prediction accuracy and decision quality in traditional two-stage methods (TSM), and (2) the bias-variance dilemma arising from the use of OBS and RCT data. This formulation enables the unbiased OR estimator to correct learning directions from biased OBS, achieving optimal bias-variance tradeoff.", "questions": "I have mentioned some questions in the weakness part above. Below are some more:\n\n I am very curious about how the data quantity and quality influences the results.  Could you discuss the convergence properties of this approach, especially when dealing with noisy or high-variance RCT data? What could happen if the method handle extreme cases where RCT data is extremely scarce (e.g., <1% of the dataset)? Are there safeguards against overfitting?", "limitations": "See above.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strength\n\n1. I like the problem itself very much. The mismatch between estimating and decision-making, from my perspective, is one of the central questions in causal inference.\n2. The bi-level optimization idea is insightful, together with the observation on the role of OBS and RCT data.The use of implicit differentiation for bi-level optimization is particularly interesting and novel.\n3. The experimentation on multiple datasets validates the framework.\n\n# Weakness\n1. I am a bit worried about the theoretical ground for conducting inference. The framework itself is sophisticated, but the settings the authors care about requires very solid theoretical results for inference (e.g., central limit theorem results). I am not super clear how to do that Bi-DFCL.\n\n2. While the paper addresses scalability, the bi-level optimization and implicit differentiation steps may still be computationally expensive for very large-scale applications. A more detailed discussion of runtime and resource requirements would be helpful.\n\n3. The experiments focus on MTBAP, where we can easily have something like eq. (1) to define $\\mathcal{L}_{DL}(\\theta)$. Is it always possible to easily get the counterpart of eq.(1) for other problems?", "was_revised": true, "final_justification": "The authors carefully address my concerns on theoretical and computational results. What I really appreciate is that the authors honestly acknowledged their tiny weakness.", "rebuttal": "We sincerely appreciate your approval of the idea and the novelty of this work, and thank you for the helpful suggestions. Below, we hope to address your concerns and questions to improve the clarity and readability of our paper.\n\n---\n\n### **【W1】The theoretical ground of Bi-DFCL**\n\n**[W1]: ”I am a bit worried about the theoretical ground for conducting inference. The framework itself is sophisticated, but the settings the authors care about requires very solid theoretical results for inference. I am not super clear how to do that Bi-DFCL.”**\n\n**[W1Response]:** We will explain the framework of Bi-DFCL and its theoretical foundations from two perspectives: offline training and online inference.\n\n1. **Offline Training**. To address the bias-variance dilemma and prediction-decision misalignment, we formulate a bi-level optimization problem using low-variance OBS data and unbiased RCT data: Find the optimal bridge network that debiases OBS data to enable the target network (lower-level) to minimize decision loss on RCT data (upper-level). This formulation is naturally and straightforwardly defined, requiring no additional theoretical assumptions. The primary technical challenges arise in gradient computation for the bridge network during upper-level optimization $\\nabla\\_{\\phi}\\mathcal{L}\\_{\\text {DL }}\\left(\\theta^{\\star}(\\phi) ; \\mathcal{D}\\_{\\mathrm{RCT}}\\right)$, specifically:\n\t1. **Gradient of Decision Loss**. As decision-making involves discrete optimization, the decision loss is non-differentiable. In Section 4.2, we resolve this via: a) Continuous relaxation using softmax functions. b) Black-box perturbation methods for gradient estimation.\n\t2. **Gradient of Target Network $\\theta^*(\\phi)$ w.r.t. Bridge Network $\\phi$.** No closed-form solution exists for the target network $\\theta^*(\\phi)$, making direct gradient computation infeasible. In Section 4.3, We address this by: a) Implicit differentiation for gradient derivation. b) Conjugate gradient methods to ensure computational efficiency.\n\n2. **Online Inference**. After offline training, the target network—whose architecture and parameters are identical to conventional OBS data models—is deployed directly for online inference.\n\n---\n\n### **【W2】The computational overhead**\n\n**[W2]: ”While the paper addresses scalability, the bi-level optimization and implicit differentiation steps may still be computationally expensive for very large-scale applications. A more detailed discussion of runtime and resource requirements would be helpful.”**\n\n**[W2Response]:** Thanks for this helpful advice.\n\n* **Space Efficiency**: Details of space efficiency and resource requirements can be find in **Appendix B.1.3 Experimental Details**, Our proposed Bi-DFCL does not incur additional space overhead compared to existing baselines. \n\n*  **Time Efficiency**: \n\n\t* **Online Inference:**  We only use the well-trained Target Model (see the Output of Algorithm 1) for online inference. Therefore, the inference time of our method is exactly the same as that of simple causal learning methods—because their model structures are the same. \n\n\t* **Offline Training:**\n\n\t\t* However, Bi-DFCL does introduce additional time overhead during offline training—which is one of the issues we have been working to address.\n\n\t\t1. **Experiments**\n\n\t\t\t  * As suggested by the reviewer, we compare the training time (minutes) of different baselines on Marketing data II. \n\n\t\t\t* | Dataset II |        Methods         | Training time |\n\t\t\t\t| :--------: | :--------------------: | :-----------: |\n\t\t\t\t|  RCT+OBS   |         TSM-SL         |    42.332     |\n\t\t\t\t|  RCT+OBS   |        KD-Label        |    67.358     |\n\t\t\t\t|  RCT+OBS   |         LTD-DR         |    492.559    |\n\t\t\t\t|  RCT+OBS   |       AutoDebias       |    397.886    |\n\t\t\t\t|  RCT+OBS   |    **Bi-DFCL-PPL**     |  **265.263**  |\n\t\t\t\t|  RCT+OBS   |    **Bi-DFCL-PIFD**    |  **294.927**  |\n\t\t\t\t|  RCT+OBS   | Bi-DFCL-PPL-wo ID Alg  |    345.132    |\n\t\t\t\t|  RCT+OBS   | Bi-DFCL-PIFD-wo ID Alg |    427.515    |\n\n\t\t\t  * Note: For fairness, all methods were fully trained for 500 epochs using the same model structure (no early stopping).\n\n\t\t\t* \"Bi-DFCL-wo ID Alg\" indicates that the solution to BLO problem no longer uses the implicit differentiation algorithm, but instead uses the more common explicit differentiation algorithm (see Appendix A.4 for details).\n\n\t\t2. **Theoretical Analysis**\n\n\t\t\t  * We decompose the additional time overhead of Bi-DFCL compared to TSM-SL into two parts: (1)the Decision Loss; (2)solving the BLO;\n\n\t\t\t  * Note that most of the time overhead of Bi-DFCL comes from solving the BLO. \n\n\t\t\t  * To accelerate this process, we use implicit differentiation and the CG algorithm. The latter efficiently solves large-scale linear systems numerically, avoiding matrix inversion and reducing time complexity from O(n³) to O(n).\n\n\t\t3. **Summary**\n\n\t\t\t  * Bi-DFCL does not incur additional space overhead or inference time overhead, but it does have a training time overhead 6-7 times that of the baseline TSM-SL. \n\n\t\t\t  * Bi-DFCL achieve a linear increase in training time with the number of samples.\n\n\t\t\t  * However, on the one hand, the extra offline training time is absolutely worthwhile in exchange for higher online decision quality. On the other hand, we have also made several improvements to address this issue (such as the CG algorithm).\n\n---\n\n### **【W3】Decision Loss for other problems**\n\n**[W3]: ”The experiments focus on MTBAP, where we can easily have something like eq. (1) to define decision loss. Is it always possible to easily get the counterpart of eq.(1) for other problems?”**\n\n**[W3Response]:** We explain the question through two optimization perspectives: common resource allocation problems and more complex formulations.\n\n1. **For marketing resource allocation**:  typically formalized as knapsack problems (e.g., 0-1, multi-dimensional, or multiple-choice) — Lagrangian duality algorithms yield approximate solutions and decision loss that exhibit structural similarity to Eq.(1).\n\n2. **For more complex optimization problems** : deriving closed-form solutions like Eq.(1) is not always feasible. Crucially, this limitation does not preclude gradient computation — our black-box perturbation method (PIFD) computes the gradients of decision loss without requiring closed-form solutions.\n\n---\n\n### **【Questions】**\n\n**【Q1】: ” Could you discuss the convergence properties of this approach, especially when dealing with noisy or high-variance RCT data? What could happen if the method handle extreme cases where RCT data is extremely scarce (e.g., <1% of the dataset)? Are there safeguards against overfitting?”**\n\n**[Q1Response]:** Good questions.\n\n1. **The convergence properties where RCT data is extremely scarce**:\n\n\t* **1%~10%**\n\t\t* We have discussed in the main text and **Appendix B.3 (Details of In-depth Analysis)** the performance changes when the ratio of RCT to OBS data ranges from (1:10 to 1:100).\n\n\t\t* It can be seen that as the absolute amount of RCT data decreases, the performance of Bi-DFCL decreases to a certain extent, but it still outperforms most baseline methods.\n\t* **<1%**\n\t\t* For RCT data <1%, the key is its **absolute volume** rather than proportion, with two scenarios:\n\t\t\t1. OBS data expands but RCT data stays remains at hundreds of of thousands (e.g., 220,000)：Even though the proportion of RCT data <1%, Bi-DFCL’s performance does not deteriorate. Instead, it becomes more robust due to the increased OBS data.\n\t\t\t2. The absolute volume of RCT data continues to decrease (e.g.,  < 220,000) :\n\t\t\t\t* In experiments, we empirically set the batch size for decision loss at the tens-of-thousands level (e.g., 65,536). Thus, if there are still tens of thousands of RCT data (e.g., more than 0.1%), our method may still work (as discussed in Appendix B.3), because the decision loss constructed from tens of thousands of RCT data remains reasonably representative.\n\t\t\t\t* If the volume of RCT data is only a few thousand or less (less than 0.1%), it is inherently difficult to train a good model with such a small amount of RCT data, but we have also implemented safeguards against overfitting to prevent extremely poor performance (will be discussed in the following text).\n2. **Are there safeguards against overfitting?**\n\t* Yes, this is an insightful question. We have indeed done a lot of work to combat high variance when RCT data is scarce, and the specific measures are as follows：\n\t\t1. **The counterfactual pseudo-labels of parameterized prediction loss.**\n\n\t\t\t* Counterfactual pseudo-labels are a key method to reduce training variance.\n\t\t\t* Unlike unstable existing reweighting methods (prone to extreme weights causing high variance), our parameterized prediction loss with such pseudo-labels is more stable.\n\t\t2. **Gating coefficients for stability and ensuring a baseline.**\n\t\t\t* Via knowledge distillation, the bridge model outputs gating coefficients to adaptively weight two teacher models’ outputs for counterfactual pseudo-labels. \n\t\t\t* This gating-based integration enhances stability, improves fault tolerance, mitigates high variance, and reduces warm-start needs.\n\t\t3. **The implicit differentiation algorithm.**\n\t\t\t* Its advantages (with the CG algorithm) include more accurate/stable gradients and higher efficiency. \n\t\t\t* It stabilizes training by avoiding explicit differentiation’s gradient vanishing, addressing high variance.\n\t\t4. **Other tricks**:\n\t\t\t* Setting the model structure of the bridge network to be relatively simple (e.g., reducing the number of layers) can effectively alleviate the overfitting phenomenon.\n\t\t\t* Updating the bridge network every k steps can also mitigate high variance.\n\n---\n\n**We sincerely thank you for your feedback and will provide more clarifications and explanations in the revised version, and welcome any further technical advice or questions on this work and we will make our best to address your concerns.**"}, {"review_id": "OPKqdYg1r9", "reviewer": "Reviewer_HrqG", "summary": "The paper proposes Bi-Level Decision-Focused Causal Learning (Bi-DFCL), a novel framework for large-scale marketing optimization that bridges observational and experimental data. It addresses two key challenges: (1) prediction-decision misalignment in traditional two-stage methods and (2) bias-variance tradeoffs between biased but abundant OBS data and unbiased but scarce RCT data. Bi-DFCL introduces a bi-level optimization framework where an upper-level unbiased OR estimator dynamically corrects the lower-level ML model via a Bridge Network. The method demonstrates significant improvements over baselines in offline benchmarks and real-world A/B tests on a major food delivery platform.", "questions": "- The bi-level optimization and implicit differentiation are elegant but computationally intensive. Could you provide runtime comparisons across datasets? How does Bi-DFCL scale with the number of treatments or sample size?\n- For large-scale deployments, have you explored approximations to reduce overhead?\n- Table 3 suggests 'Implicit Differentiation' contributes marginally to gains (e.g., +0.29% in Marketing Data I). Is this due to optimization stability, or are there cases where it's critical?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The bi-level optimization design bridges OBS and RCT data, leveraging their complementary strengths. The Bridge Network and implicit differentiation algorithm are novel contributions.\n- Extensive validation on public benchmarks, industrial datasets, and online A/B tests underscores real-world applicability. \n- The paper also provides theoretical grounding, i.e., unbiased estimator proofs.\n- The paper is well-written with clear and logical structure.\n\nWeaknesses\n- The bi-level optimization and implicit differentiation may incur high computational costs for large datasets. Scalability details (e.g., runtime comparisons) are not thoroughly discussed.\n- The focus on marketing raises questions about applicability to other domains (e.g., healthcare, logistics). The method’s reliance on RCT data might limit adoption in settings where RCTs are infeasible.\n- Table 3’s incremental ablation could better isolate the impact of counterfactual labels and implicit differentiation. For instance, how much does the Bridge Network alone contribute vs. the full bi-level setup?", "was_revised": true, "final_justification": "I keep my original justification, weak accept.", "rebuttal": "We sincerely appreciate your approval of the idea and the novelty of this work, and thank you for the helpful suggestions. Below, we hope to address your concerns and questions to improve the clarity and readability of our paper.\n\n---\n\n### **【Q1】The  computational overhead**\n\n**[Q1]: ”Could you provide runtime comparisons across datasets? How does Bi-DFCL scale with the number of treatments or sample size?”**\n\n**[Q1Response]:** \n\n* **Online Inference:**  \n\n\t* We first want to clarify that for the online deployment and inference of Bi-DFCL, we only use the well-trained Target Model (see the Output of Algorithm1). \n\t* Therefore, **the inference time of our method is exactly the same as that of simple causal learning methods—because their model structures are identical**. \n\n* **Offline Training:**\n\n\t* However, Bi-DFCL does introduce additional time overhead during offline training—which is one of the issues we have been working to address. \n\n\t1. **Runtime Comparisons with different baselines**\n\n\t\t  * As suggested by the reviewer, we compare the training time (minutes) of different baselines (using RCT + OBS data for training) on the Marketing data II. The results are shown below.\n\n\t\t* | Dataset II |     Methods      | Training time |\n\t\t\t| :--------: | :--------------: | :-----------: |\n\t\t\t|  RCT+OBS   |      TSM-SL      |    42.332     |\n\t\t\t|  RCT+OBS   |     KD-Label     |    67.358     |\n\t\t\t|  RCT+OBS   |      LTD-DR      |    492.559    |\n\t\t\t|  RCT+OBS   |    AutoDebias    |    397.886    |\n\t\t\t|  RCT+OBS   | **Bi-DFCL-PPL**  |  **265.263**  |\n\t\t\t|  RCT+OBS   | **Bi-DFCL-PIFD** |  **294.927**  |\n\n\n\t\t  * Note: For fairness, all methods were fully trained for 500 epochs using the same model structure (no early stopping).\n\n\t2. **Runtime scale with the number of treatments or sample size**\n\n\t\t  * As suggested by the reviewer, we also compare the training time (minutes)  scale with the number of sample size. The results are shown below.\n\n\t\t* | Dataset II (RCT+OBS) |   Methods    | Training time |\n\t\t\t| :------------------: | :----------: | :-----------: |\n\t\t\t|         100%         |    TSM-SL    |    42.332     |\n\t\t\t|         50%          |    TSM-SL    |    20.207     |\n\t\t\t|         25%          |    TSM-SL    |    10.738     |\n\t\t\t|         100%         | Bi-DFCL-PPL  |    265.263    |\n\t\t\t|         50%          | Bi-DFCL-PPL  |    124.993    |\n\t\t\t|         25%          | Bi-DFCL-PPL  |    59.193     |\n\t\t\t|         100%         | Bi-DFCL-PIFD |    294.927    |\n\t\t\t|         50%          | Bi-DFCL-PIFD |    125.391    |\n\t\t\t|         25%          | Bi-DFCL-PIFD |    66.839     |\n\n\t\t  * From the table, the following conclusions can be drawn: \n\n\t\t\t   * Our proposed Bi-DFCL, like the baseline TSM-SL, shows a roughly **linear increase in training time as the sample size changes**.\n\n\t\t\t   * As a supplement, **the training time does not increase with the number of treatments** (the minimal increase in time is negligible).\n\n\t\t  * In addition, we **theoretically explain** why Bi-DFCL can achieve a linear increase: \n\n\t\t\t* This is because most of the training time of Bi-DFCL comes from solving the bi-level optimization problem - implicit differentiation.\n\n\t\t\t* Implicit differentiation itself requires calculating the inverse of a large-scale matrix, which theoretically has a time complexity of O(n³)  (where n is the number of model parameters). \n\n\t\t\t* However, we use a numerical method — the CG algorithm — to solve this problem, reducing the time complexity of this step from O(n³) to O(n).\n\n\t3. **Summary**\n\n\t\t* Bi-DFCL does not incur additional space overhead or inference time overhead, but it **does have a training time overhead 6-7 times that of the baseline TSM-SL**. \n\t\t* **Bi-DFCL achieve a linear increase in training time with the number of samples.**\n\t\t* However, on the one hand, the extra **offline training time is absolutely worthwhile in exchange for higher online decision quality**. On the other hand, we have **also made several improvements to address this issue (such as the CG algorithm)**.\n\n---\n\n### **【Q2】Approximations to reduce overhead**\n\n**[Q2]: ”For large-scale deployments, have you explored approximations to reduce overhead?”**\n\n**[Q2Response]:** \n\n* Yes, we have tried many ways to reduce the training time overhead of Bi-DFCL — this is one of our key goals. \n* **Online Inference (large-scale deployments):**  the inference time of our method is exactly the same as that of simple causal learning methods. \n* **Offline Training:** The following are the methods we ultimately adopted that can effectively reduce the training time overhead：\n\n\t1. **Implicit differentiation and CG algorithm for solving the BLO problems**:\n\t\t* The CG algorithm efficiently solves the following large-scale linear system through numerical methods, thereby avoiding matrix inversion — reducing the time complexity from O(n³) to O(n).\n\n\t2. **Hyperparameter—*k* (number of GD steps for assumed updates)**:\n\n\t\t* Implicit differentiation, as it does not depend on the optimization path, can obtain a more accurate optimal solution compared to explicit differentiation(assuming one step GD to reach optimal). This allows solving the bi-level optimization problem once every k batches, rather than having k=1 as in explicit differentiation.\n\n\t\t* We use k=5 by default, which eliminates the need to solve the BLO problem every time, thereby significantly reducing the time overhead.\n\n\t3. **Gating coefficients, warm starts to accelerate convergence:**\n\n\t\t* We have done a lot of work to reduce the training variance of Bi-DFCL — for example, gating coefficients are used to generate more stable counterfactual pseudo-labels, and warm starts are used to accelerate convergence. \n\n\t\t* These efforts are aimed at training stability, which can greatly accelerate the convergence of BLO problems, thereby reducing time overhead.\n\n---\n\n### **【Q3】Contributions of Implicit Differentiation**\n\n**【Q3】: ”Table 3 suggests 'Implicit Differentiation' contributes marginally to gains (e.g., +0.29% in Marketing Data I). Is this due to optimization stability, or are there cases where it's critical?”**\n\n**[Q3Response]:** \n\n* Yes, the advantages of Implicit Differentiation (and the CG algorithm used therein) mainly include three aspects: **more accurate gradients; more stable gradients; and higher solution efficiency**. \n* The contributions of Implicit Differentiation shown in Table 3 (such as 0.29% and 0.39%) stem from its first advantage: more accurate gradients.\n* However, **the other two advantages are also very important (especially gradient stability)**.\n* The commonly used existing solution is the explicit differentiation algorithm(see Appendix A.4 for details). We summarize the advantages of Implicit Differentiation as follows:\n\t* **more accurate gradients**：\n\t\t* Explicit Differentiation Algorithm relies heavily on the optimization path and, when combined with decision loss, is often susceptible to vanishing gradients and suboptimal solutions.\n\t\t* It should be emphasized that the assumption of reaching the optimum in one gradient descent step is often unrealistic. In practice, a single update typically leads to a suboptimal solution, whereas multiple updates can result in severe vanishing gradient issues. \n\t* **more stable gradients**：\n\t\t* As introduced above, the vanishing gradient problem often occurs in the explicit differentiation algorithm. \n\t\t* In practice, we found that its training process is very unstable, so it is necessary to print gradient information for checking every update, and the experimental results are the optimal results when the vanishing gradient problem is not serious. \n\t\t* That is to say, the results of our implicit differentiation algorithm are more stable with smaller variance.\n\t* **higher solution efficiency**：\n\t\t* As discussed in Q2, the CG algorithm efficiently solves large-scale linear systems numerically, avoiding matrix inversion and reducing time complexity.\n\n---\n\n**【W2】: Domains with no experimental data**\n\n**[W2Response]:** \n\n- We acknowledge that there are some causal inference scenarios where no RCT data is available. However, it is regrettable that this paper assumes the domain of marketing optimization (where a small amount of RCT data can be obtained).\n- Meanwhile, we believe that Bi-DFCL has certain value in other scenarios beyond marketing optimization where RCT data can be obtained even when extremely scarce. We conducted experiments in **Appendix B.3 (Details of In-depth Analysis), Table 6**, which demonstrate that our method still has high application value when RCT data is extremely sparse (e.g., only 1%).\n- However, scenarios without any available RCT data are beyond the scope of this paper, but this will be our future research direction. We apologize for this and hope the reviewers will understand. We greatly appreciate the pointing out of this issue.\n\n---\n\n**【W3】:  How much does the Bridge Network alone contribute vs. the full bi-level setup?**\n\n**[W3Response]:** Thanks for this question. We apologize for the misunderstanding caused.\n\n* First, we clarify that the bridge model and the BLO are integrated, that is, **the column of the bi-level optimization problem in the table 3 of ablation study represents the contribution of the bridge model.**\n* Secondly, due to space constraints, we have placed the description of ablation study in **Appendix B.2 Details of Ablation Studies.** \n* We apologize for the ambiguity here, and in subsequent versions, we will move the detailed description of the ablation study in Appendix B.2 to the main text to facilitate better understanding for readers.\n\n---\n\n**We sincerely thank you for your feedback and will provide more clarifications and explanations in the revised version, and welcome any further technical advice or questions on this work and we will make our best to address your concerns.**"}]

Author rebuttal:

Meta-review: The paper introduces a framework for addressing the prediction-decision misalignment in marketing optimization, bridging both observational and experimental data. The paper was well-received by most of the reviewers and inspired (what looks like) a fruitful exchange between the reviewers and the authors who addressed the reviewer concerns. One reviewer increased their score after the rebuttal. To meet the expectations of the reviewers, the authors should incorporate the rebuttal in their final paper, including improving the clarity of the paper, incorporating the computational cost results, and moving some of the sensitivity analysis to the main paper.
Decision: Accept (poster)