Conformal Prediction for Dose-Response Models with Continuous Treatments

## Abstract

Abstract Understanding the dose-response relation between a continuous treatment and the outcome for an individual can greatly drive decision-making, particularly in areas like personalized drug dosing and personalized healthcare interventions. Point estimates are often insufficient in these high-risk environments, highlighting the need for uncertainty quantification to support informed decisions. Conformal prediction, a distribution-free and model-agnostic method for uncertainty quantification, has seen limited application in continuous treatments or dose-response models. To address this gap, we propose a novel methodology that frames the causal dose-response problem as a covariate shift, leveraging weighted conformal prediction. By incorporating propensity estimation, conformal predictive systems, and likelihood ratios, we present a practical solution for generating prediction intervals for dose-response models. Additionally, our method approximates local coverage for every treatment value by applying kernel functions as weights in weighted conformal prediction. Finally, we use a new synthetic benchmark dataset to demonstrate the significance of covariate shift assumptions in achieving robust prediction intervals for dose-response models.

## 1 Introduction

How can we determine the optimal dose for a patient to ensure the best therapeutic outcome? What is the impact of discounts in an online store on sales? What impact does $CO_{2}$ concentration have on local climates? At the core of each of these questions lies a shared causal idea: understanding the dose-response relation under continuous treatments to inform decision-making. In many cases, these decisions bear significant consequences, where relying solely on point estimates may be insufficient (Feuerriegel et al., ). Particularly in high-stakes situations, augmenting predictions with uncertainty quantification (UQ) can significantly improve decision-making processes (Feuerriegel et al., ). For instance, while the estimated causal effect of a continuous treatment may appear positive, prediction intervals could suggest a largely negative outcome for a specific individual. Such insights are crucial for deciding interventions. To tackle this, conformal prediction (CP) offers a robust solution for UQ, being both distribution-free and model-agnostic, with formal coverage guarantees (Vovk et al., ).

In this work, we seek to extend CP to UQ in dose-response models, aiming to aid decision-makers with more informed estimates to tackle such questions. We introduce a novel approach for deriving prediction intervals in the continuous treatment setting using weighted conformal prediction by combining propensity estimation with weighted conformal predictive systems. Furthermore, with the aid of a novel synthetic benchmark, we show how viewing the problem as a covariate shift approach provides coverage across all treatment values to help create more individualized dose-response curves.

## 2 Background

In this paper we expand upon the potential outcomes framework introduced in Rubin (), otherwise known as the Rubin framework to accommodate continuous treatments. Consider a continuous treatment variable $T\in[t_{L},t_{U}]$ with a lower bound $t_{L}$ and upper bound $t_{U}$, observed covariates $X$, and potential outcomes $Y(t)\in\mathbb{R}$ representing the outcome that would be observed under treatment level $t$. The Conditional Average Dose-Response Function (CADRF) is defined as $\nu(x,t)=E[Y(t)|X=x]$, the expected value over the Individual Dose-Response Functions (IDRF) for all individuals with observed $X$. Similar to Conditional Average Treatment Effects (CATE), to estimate the CADRF we make the following standard assumptions (Rubin, ; Hirano & Imbens, ):

- •
Unconfoundedness: $Y(t)\perp\!\!\!\perp T|X,\,\forall t\in T$. This assumption states that, conditional on the observed covariates, the treatment assignment is independent of the potential outcomes. In other words, there are no unobserved confounders that influence both the treatment assignment and the outcome.
- •
Overlap or positivity: $0<P(T=t|X=x)<1,\,\forall t\in T$ with $x\in X$. The overlap assumption ensures that for every covariate value $x$, there is a positive probability of receiving any treatment level. This is crucial for estimating treatment effects across the entire range of treatment levels.
- •
Consistency: $Y=Y(t)$ with probability $1$. This assumption links the observed outcomes to the potential outcomes, stating that the observed outcome is equal to the potential outcome corresponding to the treatment received.

Quantifying the IDRF requires observing the $Y(t)$ for all possible treatment values. These treatment values are all counterfactuals and thus impossible to observe as we only can observe $Y$ for a single treatment value $t$ at a time. Furthermore for estimating the CADRF, likewise with CATE estimation, the distribution of the treatment assignment can bias the estimation (Hirano & Imbens, ). This distribution of the treatment assignment is called the propensity distribution, which was initially defined for binary treatments. Hirano & Imbens () introduced the generalized propensity score (GPS) for continuous treatments that aims to unbias the CATE estimation for continuous treatments. The GPS is defined as $\pi(t_{i}|x)=f_{T|X}(T=t_{i}|X=x)$, which is the evaluation of $T=t_{i}$ on the conditional probability density function $T|X$ (Hirano & Imbens, ). If the treatment is independent of $X$, i.e. there are no confounders that influence treatment assignment, then $f_{T|X}$ is equal for all possible $X$. Furthermore, the treatment assignment is considered uniformly assigned between lower $t_{L}$ and upper $t_{U}$ possible treatment if $f_{T|X}$ represents the density function of the uniform distribution between $t_{L}$ and $t_{U}$. The GPS can then be used to mimic the randomly assigned treatment to estimate the unbiased CADRF (Wu et al., ).

The simplest method to estimate the CADRF is using an S-learner where a single learner is fit on both the covariates $X$ and the treatment $T$ to estimate $Y$. This approach provides a CADRF for each specific sample by keeping the covariates $X$ constant and changing $T$ to all different treatment values. However, if the treatment in the data is not uniformly assigned then the epistemic error can increase for specific treatment values $t_{i}$ and $X=x$ in low overlap regions or where $\pi(t_{i}|x)$ becomes very small. Consequently inferring $T=t_{i}$ in these regions would yield unreliable model estimates which should be communicated to ensure correct usage of a CADRF model.

The estimated $\widehat{IDRF}$ can also be seen as follows: $\widehat{IDRF}=\nu(x,t)+\epsilon_{a,IDRF}(x,t)+\epsilon_{e,IDRF}(x,t)$. The aleatoric uncertainty is symbolized by $\epsilon_{a,IDRF}(x,t)$ created by the inherent variability between individuals having the same covariates. $\epsilon_{e,IDRF}(x,t)$ symbolises the epistemic uncertainty coming from model specification and finite samples. Estimating both uncertainties creates the opportunity to estimate the ranges of the $\widehat{IDRF}$:

##### Problem Definition

To accurately estimate the $\widehat{IDRF}$ for all possible treatment values we require correctly estimating both uncertainties for all treatment values equally, or more formally; for a specific significance level $\alpha$, lower treatment bound $t_{L}$, upper treatment bound $t_{U}$, and covariates $X$, we require prediction intervals $C(t,X)$ such that

$$ $\mathbb{P}(Y(t)\in C(X,t))\geq 1-\alpha,\quad\forall t\in[t_{L},t_{U}]$ (1) $$

This requirement necessitates prediction intervals that guarantee coverage for each possible treatment value individually.

## 3 Related Work

Our proposed solution combines three different domains: propensity score methods, conformal prediction, and treatment effect or dose-response modelling.

Propensity score methods, introduced by Rosenbaum & Rubin (), have become widespread in causal inference, especially in observational studies. These methods aim to balance confounders across treatment groups, reducing bias in treatment effect estimates. Hirano & Imbens () generalized this propensity score to continuous instead of binary treatments, introducing the generalized propensity score and building the foundation for causal inference with continuous exposures. Wu et al. () used the generalized propensity score for matching continuous treatments to debias the treatment assignment and more accurately estimate the average dose-response curve for all treatment values. Other approaches adapt machine learning techniques to dose-response modelling. For instance, Athey et al. () developed generalized random forests for heterogeneous treatment effect estimation, adaptable to continuous treatments.

To provide UQ, this work adapts conformal prediction. Conformal prediction is a model-agnostic method introduced by Vovk et al. () that constructs prediction intervals with guaranteed finite-sample coverage under distribution-free assumptions. Conformal prediction uses conformity scores to assess uncertainty. Various improvements, such as the adaptive version by Romano et al. (), have increased the flexibility and applicability to even heteroscedastic settings. Additionally, Lei et al. () and Papadopoulos et al. () introduced split conformal prediction, significantly improving computational efficiency. For scenarios involving covariate or distribution shifts, Tibshirani et al. () introduced weighted conformal prediction to ensure coverage under mismatched training and testing data distributions, with additional work by Gibbs & Candes (; ) and Barber et al. (). By reweighting the calibration samples similar to weighted conformal prediction,  Guan () introduced localized conformal prediction where the prediction intervals are determined by calibration samples localized around the test sample. Vovk et al. () also introduced conformal predictive systems (CPS); an extension of full conformal prediction that allows extracting predictive distributions instead of prediction intervals. More recently, Jonkers et al. () combined previous concepts, introducing weighted conformal predictive systems to also account for covariate shifts.

In causal inference, conformal prediction has mainly been applied to binary treatments. For instance, Lei & Candès () were among the first to apply conformal prediction to treatment effects estimation in randomized experiments and confounded or observational data. Jonkers et al. () and Alaa & Ahmad () extended this approach to the potential outcomes framework, providing uncertainty to quantify individual treatment effects. However, the use of conformal prediction in continuous treatment settings remains largely unexplored. Schröder et al. () proposed a conformal prediction framework for prediction intervals of treatment effects for continuous treatment interventions. However, their approach mainly covers single-treatment interventions and is computationally intensive, requiring optimization per confidence level, treatment, and sample where they provide prediction intervals for a single treatment value. For a more in-depth analysis of Schröder et al. (), see Appendix C.

Our goal is to achieve predictive coverage across the entire range of the treatment variable in estimating the dose-response curve. To our knowledge, no existing UQ methods offer conformal prediction guarantees for dose-response models with continuous treatments. To address this gap, we propose a novel methodology that seeks to provide this coverage by integrating weighted conformal prediction with propensity score weighting thereby guaranteeing coverage for any treatment value in continuous treatment dose-response models.

## 4 Method

### 4.1 Introduction to Conformal Prediction

Before delving into our proposed method, we provide a formal introduction to conformal prediction (Jonkers et al., ; Tibshirani et al., ). Conformal prediction offers a powerful method for constructing prediction intervals with guaranteed finite-sample coverage under distribution-free assumptions (Vovk et al., ). The key insight of conformal prediction lies in its use of a nonconformity measure to quantify the degree to which a new observation differs from previously observed data.

Let us consider a regression problem with the training data being $n$ independent and identically distributed (i.i.d.) data pairs $Z_{1}=(X_{1},y_{1}),...,Z_{n}=(X_{n},y_{n})$, where $X_{i}\in\mathbb{R}^{d}$ represents a vector of $d$ features and $y_{i}\in\mathbb{R}$ the corresponding label. Consider $Z_{n+1}=(X_{n+1},y_{n+1})$ a new exchangeable point being the test observation to evaluate and provide prediction intervals. Conformal prediction aims to construct a prediction interval $\hat{C}(X_{n+1})$ such that

$$ $\mathbb{P}\{y_{n+1}\in\hat{C}(X_{n+1})\}\geq 1-\alpha$ (2) $$

for a pre-specified significance level $\alpha\in(0,1)$ where the probability is calculated over the points $Z_{i},i=1,...,n$ .

To achieve this, we first define a nonconformity measure $S((X,y),Z_{1:n})$ that quantifies how different the pair $(X,y)$ is from a multiset $Z_{1:n}=\{Z_{1},...,Z_{n}\}$ of data points. The lower the nonconformity measure, the more the pair conforms to the multiset $Z_{1:n}$. The most commonly used nonconformity measure is the absolute error $S((X,y),Z_{1:n})=|y-\hat{\mu}(X)|$ with $\hat{\mu}$ an estimator fitted on $Z_{1:n}$.

Next, for each possible value $y\in\mathbb{R}$ that $y_{n+1}$ could be, we compute the nonconformity scores:

$$ $R_{i}^{y}:=S((X_{i},y_{i}),\{(X_{1},y_{1}),...,(X_{i-1},y_{i-1}),(X_{i+1},y_{i +1}),...,(X_{n},y_{n}),(X_{n+1},y)\}),i=1,...,n$ (3) $$

$$ $R_{n+1}^{y}:=S((X_{n+1},y),\{(X_{1},y_{1}),...,(X_{n},y_{n})\})$ (4) $$

Finally, we construct the prediction interval containing all $y$ where (Jonkers et al., )

$$ $\hat{C}(X_{n+1})=\bigg{\{}y\in\mathbb{R}:\frac{\#\{i=1,...,n+1:R_{i}^{y}\geq R _{n+1}^{y}\}}{n+1}\geq 1-\alpha\bigg{\}}$ (5) $$

Tibshirani et al. () presented conformal prediction slightly differently by using quantile functions instead, which will be more convenient for weighted conformal prediction later on.
Tibshirani et al. () defines the $1-\alpha$ quantile function as follows, where $F_{R}(y)$ represents the distribution of nonconformity scores $R_{i}^{y}$ consisting of a sum of point masses $\delta_{a}$ with mass at $a$ where $R^{y}\sim F_{R}(y)$ (Tibshirani et al., ). $F_{R}(y)$ can then be used to calculate probabilities:

$$ $Quantile(1-\alpha;F_{R}(y))=inf\{R_{i}^{y}:\mathbb{P}\{R^{y}\leq R_{i}^{y}\} \geq 1-\alpha\}$ (6) $$

$$ $F_{R}(y)=\frac{1}{n+1}\sum_{i=1}^{n}\delta_{R_{i}^{y}}+\frac{1}{n+1}\delta_{\infty}$ (7) $$

Finally, we construct the prediction interval containing all $y$ where

$$ $\hat{C}(X_{n+1})=\{y\in\mathbb{R}:R_{n+1}^{y}\leq Quantile\left(1-\alpha;F_{R} (y)\right)\}$ (8) $$

This procedure guarantees that $P(y_{n+1}\in\hat{C}(X_{n+1}))\geq 1-\alpha$ for any exchangeable distribution of the data and any choice of nonconformity measure (Tibshirani et al., ).

#### 4.1.1 Inductive Conformal Prediction

The previously mentioned conformal prediction approach is computationally heavy as it requires fitting $n\cdot\#\{\mathbb{R}\}+1$ estimators $\hat{\mu}$. Inductive or split conformal prediction (ICP), introduced by Papadopoulos et al. (), tackles this computation issue by splitting the training sequence $Z_{1:n}=\{Z_{1},...,Z_{n}\}$ into two sets: the proper training set $Z_{1:m}=\{Z_{1},...,Z_{m}\}$ and the calibration set $Z_{m+1:n}=\{Z_{m+1},...,Z_{n}\}$. A single regression model $\hat{\mu}$ is fit on the proper training set while the nonconformity scores (e.g., $R_{i}=|y_{i}-\hat{\mu}(X_{i})|,i=m+1,...,n$) are generated from the calibration set. These scores are sorted in descending order denoted as $R_{1}^{*},...,R_{n-m}^{*}$. Then, for a new sample with features $X_{n+1}$, a point prediction is made $\hat{y}_{n+1}=\hat{\mu}(X_{n+1})$. Finally, given a target coverage of $1-\alpha$, the prediction interval becomes

$$ $\hat{C}(X_{n+1})=[\hat{y}_{n+1}-R_{s}^{*},\hat{y}_{n+1}+R_{s}^{*}]$ (9) $$

where $s=\lfloor\alpha(n-m+1)\rfloor$ represents the $1-\alpha$ quantile of the ordered nonconformity set with size $n-m$ (Jonkers et al., ).

#### 4.1.2 Weighted Conformal Prediction

Evaluating and requiring coverage guarantees for the dose-response model at all possible treatment values changes the test distribution compared to the training distribution. In the training data, all treatment values are sampled according to their (conditional) training distribution, which can be determined by other variables in the case of confounding. However, every treatment value is possible in testing, and thus, every treatment sample can be sampled. This mimics sampling a new test sample with the treatment value from a uniform distribution, which can be vastly different from the treatment distribution in the training data. Standard conformal prediction only guarantees coverage if the joint distribution of the new sample $Z_{n+1}$ and $Z_{1:n}$ remains the same under permutations, which is called the exchangeability assumption (Vovk et al., ; Tibshirani et al., ). This issue is called covariate shift; The features $X_{n+1}$ come from a different distribution compared to $X_{1:n}$, while the relation between $X$ and $y$ remains the same. More formally: $X_{i}\sim P_{X},\ i=1,...,n$ and $X_{n+1}\sim\tilde{P}_{X}$ where $\tilde{P}_{X}\neq P_{X}$ while $y_{i}\sim P_{Y|X},\ i=1,...,n$.

Weighted conformal prediction provides a solution to tackle this issue (Tibshirani et al., ). However, their main assumption is that the likelihood ratio between the training $P_{X}$ and the test covariate distribution $\tilde{P}_{X}$ is known, defined as

$$ $w(x)=\frac{d\tilde{P}(x)}{dP(x)}$ (10) $$

The rationale is that they reweight the distribution of nonconformity scores $F_{R}(y)$ to make the nonconformity scores more exchangeable with the test population by using the following weights in equation 7 (Tibshirani et al., ):

$$ $p^{w}_{i}(X_{n+1})=\frac{w(X_{i})}{\sum_{j=1}^{n}w(X_{j})+w(X_{n+1})}\qquad p^ {w}_{n+1}(X_{n+1})=\frac{w(X_{n+1})}{\sum_{j=1}^{n}w(X_{j})+w(X_{n+1})}$ (11) $$

$$ $F_{R}(y)=\sum_{i=1}^{n}p_{i}^{w}(X_{n+1})\delta_{R_{i}^{y}}+p_{n+1}^{w}(X_{n+1 })\delta_{\infty}$ (12) $$

Consequently, these weights adjust the distribution of nonconformity scores to give more weight to nonconformity scores that are more likely in the test set and vice versa while in standard conformal prediction, every $R_{i}$ has equal weight. Also, note that the weights $p^{w}(x)$ are normalized, cancelling out any constant terms resulting in $w(x)$ being proportional to $w(x)\propto\frac{d\tilde{P}(x)}{dP(x)}$. An extension to split weighted conformal prediction can be done similarly as in section 4.1.1 (Tibshirani et al., ).

#### 4.1.3 Conformal Predictive Systems

In some cases, providing a prediction interval often does not suffice and a complete predictive distribution is required. The extension proposed by Vovk et al. () produces a predictive distribution by arranging p-values, created using specific conformity measures, into a probability distribution function. A requirement to create a Conformal Predictive System (CPS) is to use a specific type of conformity measures (^1^11For the specific definition see Vovk et al. ()) which include monotonic measures. Then, given the training data $Z_{1:n}$ and observed test sample $X_{n+1}$, we define an example of this specific conformity measure $S$ and conformity scores $R_{i}^{y}$ similar as in equations 3 and 4:

$$ $S((X,y),Z_{1:n})=y-\hat{\mu}(X)$ (13) $$

With $\hat{\mu}$ an estimator fitted on the training set $Z_{1:n}$. $R_{i}^{y}$ and $R_{n+1}^{y}$ are then similarly defined as in equation 3 for a CPS. Then, as defined in Vovk et al. () we can define a predictive distribution Q for value $y$, using a distribution of nonconformity scores $F_{R}(y)$ of $y$ to calculate $\mathbb{P}$, similarly to the quantile function in equation 6 as follows:

$$ $Q_{R}(y,\phi)=\mathbb{P}_{F_{R}(y)}\{R^{y}<R_{n+1}^{y}\}+\phi\cdot\mathbb{P}_{ F_{R}(y)}\{R^{y}=R_{n+1}^{y}\}$ (14) $$

Where $\phi$ is a random number sampled from a uniform distribution between 0 and 1 to ensure a smooth predictive distribution. Using the same approach as section 4.1.2, these conformal predictive systems can be expanded to weighted conformal predictive systems by adjusting $F_{R}(y)$ to account for the covariate shift (Jonkers et al., ).

Additionally, conformal predictive systems also suffer from computational issues, therefore Vovk et al. () introduced split conformal predictive systems to tackle the same issues in a way analogous to section 4.1.1.

### 4.2 Proposed methodology: Propensity Weighted Conformal Prediction

Taking into account the background knowledge of conformal prediction, we first need to formally define the target distribution to tackle our problem definition. A CADRF model $\hat{\nu}(X,T)$ is trained on triples $(X,T,Y)$ with X $d$-dimensional observed covariates $X\in\mathbb{R}^{d}\sim P_{X}$ and continuous treatment variables $T\in[t_{L},t_{U}]\sim P_{T|X}$ to predict responses $Y\in\mathbb{R}\sim P_{Y|T,X}$. $P_{X}$ represents the covariate distribution, $P_{T|X}$ represents the observational conditional treatment distribution given confounders $X$, and $P_{Y|T,X}$ represents the outcome distribution. $P_{T|X}=P_{T}$ if there are no confounders for T. A CADRF model will be used to query the dose-response for all $T\in[t_{L},t_{U}]$, creating an interventional distribution $\tilde{P}_{T}$. As every treatment value $t$ is equally likely we can define $\tilde{P}_{T}=\tilde{P}_{T|X}=Uniform(t_{L},t_{U})$.

To attain marginal coverage across the interventional test set for a CADRF we can use weighted conformal prediction (Tibshirani et al., ). This requires defining the weights $w$ for $X_{i}$ and treatment value $t$ using equation 11, which we will call the global ($g$) propensity ($p$) weights $w_{g,p}$:

$$ $\displaystyle w_{g,p}(X_{i},T_{i})$ $\displaystyle=\frac{d\tilde{P}_{X,T}(X_{i},T_{i})}{dP_{X,T}(X_{i},T_{i})}= \frac{d\tilde{P}_{T|X}(X_{i},T_{i})d\tilde{P}_{X}(X_{i})}{dP_{T|X}(X_{i},T_{i} )dP_{X}(X_{i})}=\frac{d\tilde{P}_{T|X}(X_{i},T_{i})dP_{X}(X_{i})}{dP_{T|X}(X_{ i},T_{i})dP_{X}(X_{i})}$ (15) $\displaystyle=\frac{d\tilde{P}_{T|X}(X_{i},T_{i})}{dP_{T|X}(X_{i},T_{i})}= \frac{f_{U(t_{L},t_{U})}(T_{i})}{\pi(T_{i}|X_{i})}=\frac{\frac{\mathbbm{1}_{[t _{L},t_{U}]}(T_{i})}{t_{U}-t_{L}}}{\pi(T_{i}|X_{i})}\propto\frac{\mathbbm{1}_{ [t_{L},t_{U}]}(T_{i})}{\pi(T_{i}|X_{i})}$ $$

with $\mathbbm{1}_{[t_{L},t_{U}]}(T_{i})$ the indicator function for $T_{i}\in[t_{L},t_{U}]$.

We assume that there is no distribution shift for $X$ and thus $\tilde{P}_{X}(X_{i})=P_{X}(X_{i})$. Additionally, $f_{U(t_{L},t_{U})}$ is the probability density function for the uniform distribution. We also define the propensity function $\pi(T_{i}|X_{i})$ as the probability density function for $P_{T|X}(T_{i})$ as specified in Section 2. To generate the prediction intervals at treatment value $t$ for a new sample $X_{n+1}$ the weights change to $w_{g,p}(X_{n+1},t)=\frac{1}{\pi(t|X_{n+1})}$. According to the weighted exchangeability defined in (Tibshirani et al., ), this guarantees marginal coverage over the interventional distribution, for all $T\in[t_{L},t_{U}]$, and $X\sim P_{X}$.

Tibshirani et al. () also suggested a method to attain local coverage around a predetermined target point $x_{0}$ using weighted conformal prediction. Consequently, this can provide varying prediction intervals for different values of $x_{0}$ providing another heteroscedastic approach. The proposed weights, which we call the local ($l$) weights $w_{l}$, utilize kernel functions with bandwidth parameter $h$:

$$ $w_{l}^{x_{0}}(X_{i})\propto K\left(\frac{X_{i}-x_{0}}{h}\right)$ (16) $$

These weights then guarantee

$$ $\mathbb{P}_{x_{0}}\{Y_{n+1}\in\hat{C}(X_{n+1};x_{0})\}\geq 1-\alpha$ (17) $$

This assures coverage around $x_{0}$, but $x_{0}$ must be determined beforehand. Additionally, if a new $x_{0}$ must be evaluated, a new calibration procedure must be performed which should be considered when applying it to general regression use cases. However, for this work, the target interventional treatment distribution is known in advance and can all be computed before deployment. Consequently, for a target treatment value $t$ we can define $w_{l}^{t}(T_{i})\propto K(\frac{T_{i}-t}{h})$ instead.

The local weights guarantee coverage where $d\tilde{P}_{T}(T_{i})/dP_{T}(T_{i})\propto K(\frac{T_{i}-t}{h})$. To adjust the local weights for a CADRF model we need to be aware of the covariate shift introduced by evaluating the interventional distribution and thus must combine $w_{g,p}$ with $w_{local}$ to achieve weighted exchangeability. These new weights are defined as $w_{l,p}$ for target treatment $t$:

$$ $w_{l,p}^{t}(X_{i},T_{i})\propto\frac{\mathbbm{1}_{[t_{L},t_{U}]}(T_{i})K\left( \frac{T_{i}-t}{h}\right)}{\pi(T_{i}|X_{i})}$ (18) $$

To generate the prediction intervals for target treatment $t$ for a new sample $X_{n+1}$ the weights are then $w_{l,p}^{t}(X_{n+1},t)=\frac{\mathbbm{1}_{[t_{L},t_{U}]}(T_{i})K((t-t/h))}{\pi
(t|X_{i})}=\frac{\mathbbm{1}_{[t_{L},t_{U}]}(T_{i})}{\pi(t|X_{i})}$, which is equal to $w_{g,p}^{t}(X_{n+1},t)$.
By using these weights in a weighted conformal prediction framework, we provide a solution to the problem definition in Section 2.

## 5 Experiments

### 5.1 Synthetic Data

We evaluate the proposed approach on synthetic data as evaluating the true individual dose-response curve requires knowing the counterfactuals which is not feasible in real-world data.

We used three experimental setups using synthetic data, each having different scenarios that change specific parameters. Setup 1 is inspired by Wu et al. () and Setup 2 follows the experimental setup of Schröder et al. (). Both Setup 1 and 2 are clarified in Appendix A. Setup 3 is novel, proposed by us, which mimics a situation where, for every scenario, two different possible dose-response functions are possible that each depends on the covariates, resulting in heavy confounding and thus limited overlap.

For each scenario (over the different setups), 5000 samples were generated using 50 different random seeds resulting in 50 datasets for each scenario. These datasets were split into 25% test (1250), 25% calibration (1250), and 50% training (2500) samples. For each scenario, two different $\alpha$ (significance values) were evaluated (i.e., $0.1$ and $0.05$ for a confidence of $90\%$ and $95\%$ resp.). Each sample in the test set is evaluated using 40 treatment values $t_{0}$ at equal intervals between the $2\%$ and $98\%$ training treatment value quantile to include varying treatment overlap regions and to mimic the uniform treatment sampling. In the results, the coverage of all treatment values and all samples in the test set are aggregated to a single mean coverage for each experiment, resulting in 50 mean coverage results for every method and scenario.

#### 5.1.1 Setup 3

Setup 3 is a new experimental setup proposed in this work to underline the importance of compensating for confounding in UQ for CADRF. The covariates are independently sampled from a normal distribution. The treatment $T$ is confounded by two variables, determining the mean of the treatment assignment distribution:

$$ $X_{1},X_{2},X_{3}\sim\text{Normal}(0,5)\qquad T\sim\text{Normal}(X_{2}+0.1 \cdot X_{1},4)$ $$

The two scenarios have slightly different outcome distributions, as shown in Table 1. The idea is the same for both scenarios; The individual dose-response function is truly conditional and thus equal treatment values between different individuals or samples do not necessarily translate to each other. In total, there are four different possible dose-response functions depending on the covariates. Furthermore, there is heavy confounding resulting in limited samples where $T-X_{2}$ yields high values that in turn create large outcome values. This creates an opportunity for high epistemic uncertainty and limited overlap. For scenario two, the aleatoric uncertainty is also heteroscedastic based on $X_{3}$ forcing solutions to look beyond the treatment value to quantify uncertainty.

**Table 1: The outcome distributions for setup 3**
| Scenario | Outcome Distribution |
| --- | --- |
| $1$ | $Y\sim sign(X_{3})\cdot(2(T-X_{2}))^{2}+33T\cdot sign(X_{1})+\text{Normal}(0,2)$ |
| $2$ | $Y\sim sign(X_{3})\cdot(2(T-X_{2}))^{2}+33T\cdot sign(X_{1})$ |
| $+\frac{(sign(X_{3})+1)}{2}\cdot\text{Normal}(0,30)+\text{Normal}(0,2)$ |  |

### 5.2 Implementation

In the case of synthetic data, the true propensity distribution, also known as the oracle distribution, is available. However, in real-world applications, the true propensity distribution is mostly unknown. As a result, any method that relies on propensity is evaluated using both the oracle propensity distribution and an estimated propensity distribution in the experiments, denoted as "Oracle" and "Propensity" in the results respectively. The latter can be approximated by leveraging conformal prediction, specifically CPS. Do note that CPS quantifies total uncertainty and thus also includes the epistemic uncertainty while ideally only the aleatoric uncertainty is included. Additionally, this propensity distribution estimate is not completely guaranteed to be equal to the true conditional propensity distribution, which we theoretically need to get complete finite sample guarantees of validity. Although, in practice, this can still be a valid approximation. A learner is trained on the covariates $X$ to predict the treatment assignment $T$, deemed the propensity learner. Subsequently, a CPS is calibrated for this learner using the calibration set as it is more practical to extract an empirical density distribution compared to standard conformal prediction. This CPS produces an empirical density distribution being a sum of Dirac delta distribution similar to $F_{R}$, thus we require the use of kernel density estimation (KDE) to extract a continuous propensity density function for a treatment value $t$, given covariates $X_{i}$. Do note that KDE interpolates the density and depending on the KDE parameters may introduce additional epistemic error, which is a drawback of estimating the propensity in this manner. The implementation for the propensity estimation is shown in Appendix B.

For the evaluation, several methods were tested and compared, including Gaussian Process, CatBoost with Uncertainty (Duan et al., ), Standard Conformal Prediction, Locally Weighted Conformal Prediction (WCP Local), Global Propensity-Weighted Conformal Prediction (WCP Global Oracle and WCP Global Propensity), and Local Propensity-Weighted Conformal Prediction (WCP Local Oracle and WCP Local Propensity). The Gaussian Process was included in the comparison due to its widespread use for UQ in regression problems assuming a normal error distribution (Fiedler et al., ). All other approaches were based on the CatBoost model, chosen for its strong out-of-the-box performance (Dorogush et al., ). As a result, the "CatBoost with Uncertainty" method was incorporated as a baseline for comparison of UQ.

The propensity learner employed in the propensity-weighted approaches was a CatboostRegressor with 4000 iterations and default hyperparameters. Similarly, the CADRF models were based on CatBoost with 5000 iterations and default hyperparameters. The CatBoost with Uncertainty approach used the same underlying CatBoost model as the CADRF methods to ensure consistency. For the locally weighted conformal approaches, a Gaussian kernel (Theodoridis, ) was employed to represent local coverage. The bandwidth parameter for the kernel was set as $h=2\cdot(0.2\cdot\sigma_{\hat{\pi}})^{2}$, where $\sigma_{\hat{\pi}}$ denotes the standard deviation of the estimated propensity distribution(^2^22The code of the proposed methodology and the experiments are available open-source at [https://github.com/predict-idlab/dose-response-conformal-prediction](https://github.com/predict-idlab/dose-response-conformal-prediction)).

### 5.3 Results

Figure 1 presents the coverage bar plots across all methods for Setup 3 Scenario 1 on the test set. Evaluations on all other setups and scenarios can be found in Appendix D. The bar plots in Figure 1 clearly illustrate the impact of covariate shift in the treatment on coverage guarantees for methods that did not account for this shift. All propensity-weighting methods assumed uniform treatment sampling during evaluation, mimicking the interpretation of a dose-response curve for decision-making for all treatment values, keeping their coverage guarantees.

As can be seen in Figure 1, the global propensity-weighting method shows a high variance in coverage across different experiments. This variance arises due to the calibration process, which considers all possible treatment values between $t_{L}$ and $t_{U}$, including those with minimal or no overlap. Depending on the calibration and test set split, certain samples may receive a significantly large likelihood ratio, thereby assigning considerable weight to those values according to Equation 12. This inflates the size of the prediction intervals, leading to conservative estimates. The oracle estimates are also notably more conservative, as they tend to provide narrower propensity distributions. This increases the frequency of large likelihood ratios when compared to the estimated propensity distribution, where the epistemic uncertainty of the propensity learner is also taken into account by the CPS procedure. On the contrary, for a new sample, the local propensity method uses calibration samples with treatment values close to the predefined value $t_{0}$ and weighting the propensities as well. Our presented approach uses more comparable calibration samples rather than the entire dataset, resulting in more conditional prediction intervals, provided there are enough calibration samples. Our method thus combines the strengths of both the local and the propensity weighting techniques. These trends are further supported in Figure 2, which shows the prediction intervals for all weighting methods alongside the treatment assignment distribution for a specific test observation. This example highlights the necessity of the uniform treatment sampling assumption for the evaluation of dose-response curves, as both the local weighting method and standard conformal prediction produce inaccurate prediction intervals in regions with low treatment overlap. In these regions, there is insufficient data to support predictions for the model, making these predictions unreliable. Consequently, propensity-weighted methods produce much larger prediction intervals in these areas to compensate for this lack of data support. If there is almost no support or extremely low propensity values, then the propensity-weighted methods provide intervals with an infinite width to show that there is no support in these regions. It is important to note, however, that these intervals may be overly conservative if the model has indeed generalized effectively in such regions. The only way to validate this is through additional data collection in these areas to confirm the model’s performance.

Figure: (a) $\alpha=0.1$
Refer to caption: x1.png

Figure: (a) Global Propensity Weighting
Refer to caption: x3.png

Note that Schröder et al. () also introduced a conformal prediction method to provide prediction intervals in the continuous treatment setting. However, we did not include a direct comparison in this study due to the high computational complexity of their approach, which would require several years to complete the same experiments we executed in a matter of hours. For a more detailed comparison, including a discussion of the difference in assumptions and methodologies, see Appendix C.

Implementing local propensity weighting in practice is less straightforward as it involves calibrating for a set of predefined treatment values and either storing these models for later use during inference or performing this action in parallel. This has the advantage that it allows conditional prediction intervals to be calculated more quickly during inference. However, a drawback is that evaluating a treatment value not included in the predefined set requires recalibration, and must be considered for inference. Still, this approach is particularly useful in fields like drug dosing, where treatment ranges are often predefined and personalized CADRF is highly relevant or where inference of new treatment values is not time-critical. Additionally, an important factor to consider is the effective sample size $\hat{n}$ in local propensity weighting (Tibshirani et al., ; Jonkers et al., ). Reweighting $F_{R}(y)$ can significantly reduce the effective sample size, which increases variability in empirical coverage compared to standard conformal prediction. This issue is especially pronounced in regions with low treatment overlap, where the effective sample size can become extremely small. However, as prediction intervals with infinite length are possible using weighted conformal prediction, these infinite intervals additionally provide information to the user where the model cannot be trusted adding an interpretability layer to the UQ. In the current work, only an S-learner was used as a CADRF estimator which could influence the epistemic error, so in future work, more specialised dose-response models can be used to reduce the interval widths and provide even more informative prediction intervals.

Our current approach can be readily extended by incorporating other conformal prediction frameworks that support weighted conformal prediction, such as adaptive conformal prediction (Romano et al., ) or weighted conformal predictive systems (Jonkers et al., ). Additionally, the weighting can be further expanded or changed to account for other types of covariate shifts in a similar manner or serve different purposes such as evaluating interventions of causal effects, thus broadening the applicability of the proposed method.

## 6 Conclusion

In this work, we have introduced a novel approach to weighted conformal prediction for UQ in dose-response models, utilizing propensity estimation and kernel functions as weights for the likelihood ratio. Alongside a newly proposed synthetic dataset, our approach highlights the necessity of compensating for the covariate shift in the treatment assignment when evaluating dose-response models across all possible treatment values. This is achieved by assuming uniform treatment sampling during testing, similar to methods used in discrete treatment effect estimation. Additionally, by leveraging conformal predictive systems to estimate propensity distributions, we offer a practical solution to implement UQ in continuous dose-response estimation for various practical use cases.

Our contribution not only adds to the field of dose-response modelling but also facilitates delivering reliable, individualized dose-response functions. Our approach has the potential to aid decision-making for personalized dosing in fields such as marketing, policy-making, and healthcare. With this UQ for continuous treatments, we are one step closer to achieving truly personalized interventions that optimize outcomes for individuals.

#### Acknowledgments

This research was funded by the FWO Junior Research project HEROI2C which investigates hybrid machine learning for improved infection management in critically ill patients (Ref. G085920N). Jarne Verhaeghe is funded by the Research Foundation Flanders (FWO, Ref. 1S59522N). Part of this research was supported through the Flemish Government (AI Research Program).

## Appendix A Synthetic Data

### A.1 Setup 1

For setup 1, inspired by Wu et al. (), six independent covariates are sampled from various distributions representing both continuous and discrete values:

$$ $\displaystyle X_{1},X_{2},X_{3},X_{4}$ $\displaystyle\sim\text{Normal}(0,1)$ $\displaystyle X_{5}$ $\displaystyle\sim\text{Uniform}[-2,2]\text{ (Integer)}$ $\displaystyle X_{6}$ $\displaystyle\sim\text{Uniform}(-3,3)$ $$

The treatment value is confounded by all variables in this setup and thus determined by a treatment function $T_{\mu}$. All scenarios share the same treatment function except for scenario 3, where a quadratic term was added. The treatment functions are shown in Table 2.

**Table 2: The treatment functions for all scenarios in setup 1.**
| Scenario | Treatment function |
| --- | --- |
| $1,2,4,5,6,7,8$ | $T_{\mu}=-0.8+X_{1}+0.1X_{2}-0.1X_{3}+0.2X_{4}+0.1X_{5}+0.1X_{6}$ |
| $3$ | $T_{\mu}=-0.8+X_{1}+0.1X_{2}-0.1X_{3}+0.2X_{4}+0.1X_{5}+0.1X_{6}+\frac{3}{2}X_{<br>3}^{2}$ |

The true assigned treatment value $T$ is then sampled from a treatment assignment distribution to add randomness and ensure some overlap in the simulated data. This treatment assignment distribution is different for various scenarios to evaluate the differences in the assumed distributions. The various functions are shown in Table 3

**Table 3: The propensity functions per scenario for Setup 1**
| Scenario | Treatment $T$ | Treatment Assignment Distribution |
| --- | --- | --- |
| 1 | $9T_{\mu}+17$ | $\text{Normal}(0,5)$ |
| 2 | $15T_{\mu}+22$ | $\text{StudentT}(df=2)$ |
| 3 | $9T_{\mu}+15$ | $\text{Normal}(0,5)$ |
| 4 | $49\frac{e^{T_{\mu}}}{1+e^{T_{\mu}}}-6$ | $\text{Normal}(0,5)$ |
| 5 | $42\frac{1}{1+e^{T_{\mu}}}+18$ | $\text{Normal}(0,5)$ |
| 6 | $7log(|T_{\mu}|+0.001)+13$ | $\text{Normal}(0,4)$ |
| 7 | $7T_{\mu}+16$ | $\text{Normal}(0,1)$ |
| 8 | $7T_{\mu}+16$ | $20\cdot\text{Beta}(\alpha=2,\beta=8)$ |

Now, given both the covariates $X$ and the assigned treatment $T$ the outcome function is defined as a random variable sampled from a normal distribution with a variance of 5, with the mean a function dependent on both the treatment and the covariates:

$$ $\displaystyle Y$ $\displaystyle\sim-1-(2X_{1}+2X_{2}+3X_{3}^{3}-20X_{4}-2X_{5}+20X_{6})$ $\displaystyle\qquad\qquad-0.1T(1-X_{1}+X_{4}+X_{5}+X_{3}^{2})+0.13^{2}|T|^{3} sin(X_{4})+\text{Normal}(0,5)$ $$

### A.2 Setup 2

Setup 2 tests the different treatment assignment distributions in the two different scenarios, which is the same experimental setup as proposed by Schröder et al. (). The covariates are sampled from a discrete uniform distribution. The treatment is sampled from the treatment assignment distributions shown in Table 4. The outcome function is sampled from a normal distribution with a mean determined by a sinus function based on both $X$ and $T$:

$$ $\displaystyle X$ $\displaystyle\sim\text{Uniform}[1,4]\text{ (Integer)}$ $\displaystyle Y$ $\displaystyle\sim sin\left((0.05\pi)(T-X)\right)+\text{Normal}(0,0.1)$ $$

**Table 4: The propensity functions per scenario for Setup 2**
| Scenario | Treatment Assignment Distribution |
| --- | --- |
| 1 | $T\sim p\cdot\text{Uniform}(0,5X)+(1-p)\text{Uniform}(5X,40)$, $p~{}\sim\text{Bernoulli}(0.3)$ |
| 2 | $T\sim\text{Normal}(5X,10)$ |

## Appendix B Propensity Distribution Estimation

Algorithm 1 presents the propensity distribution estimation using Conformal Predictive Systems (CPS). This results in a propensity distribution array $\pi_{arr}$ with the calculated propensity density for each sample in $X_{cal}$. $exp$ is the exponential function and $len(X)$ denotes the length of the array $X$.

Figure: Algorithm 1 Estimating the Propensity Distribution

## Appendix C Comparison to Schröder et al.

In comparison to the work of Schröder et al. (), our approach differs in several key aspects. First, the aim of their work is different from ours. The aim of Schröder et al. () is to provide prediction intervals for the causal effect of treatment interventions where the treatment value is continuous. In our work, the goal is to provide prediction intervals for dose-response models instead of treatment interventions, answering a different causal question. However, adjusting our work to interventions is possible; In the case of soft interventions, the target distribution propensity changes and thus substituting the current uniform distribution in the weights $w(x)$ with the new target propensity distribution covers the soft intervention case. For hard interventions, this is an evaluation for a single treatment value which is similar to the local propensity method, but for only that target treatment value. Secondly, their approach differs in their conformal prediction approach where they want to provide correct prediction intervals for a single sample, single $\alpha$ value, and single treatment using a mathematical solver based on the proposed weighted conformal prediction by Gibbs & Candes (). Thirdly, they frame the propensity or covariate shift differently as either a Dirac distribution for a hard intervention, or a different propensity distribution in the case of a soft intervention. This is a direct consequence of their aim to quantify the causal effect of a single intervention, compared to providing a dose-response model in our case which requires a uniform assumption. Fourthly, the experimental setup of Schröder et al. () does not address the impact of a treatment covariate shift as shown by Figure 4 and Figure 5 where even standard conformal prediction (CP) achieves the required empirical coverage. Lastly, we also approach the propensity estimation in cases with unknown propensity as an uncertainty quantification problem and tackle it with conformal predictive systems. In the end, our approach offers a different solution on continuous treatment effects through dose-response modelling.

## Appendix D Additional Results

Figure: (a) $\alpha=0.1$
Refer to caption: x8.png

Figure: (a) $\alpha=0.1$
Refer to caption: x10.png

Figure: (a) $\alpha=0.1$
Refer to caption: x12.png

Figure: (a) $\alpha=0.1$
Refer to caption: x14.png

Figure: (a) $\alpha=0.1$
Refer to caption: x16.png

Figure: (a) $\alpha=0.1$
Refer to caption: x18.png

Figure: (a) $\alpha=0.1$
Refer to caption: x20.png

Figure: (a) $\alpha=0.1$
Refer to caption: x22.png

Figure: (a) $\alpha=0.1$
Refer to caption: x24.png

Figure: (a) $\alpha=0.1$
Refer to caption: x26.png

Figure: (a) $\alpha=0.1$
Refer to caption: x28.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 8
presentation: 4
questions: My questions are implied by the weaknesses listed above. I'd like to see:
- more figures akin to Figure 2
- a comparison of error of the estimated CADRF between methods
- more commentary on why the method yields such conservative prediction intervals

Additionally:
- What are the implications of the conservative prediction intervals on usefulness of the method in practical settings?
- How might the method be improved subsequently to achieve ideal empirical coverage?
soundness: 3
strengths: - Reliable dose-response estimation from observational data is important in medicine and other settings.
- The writing style, mathematical notation and presentation, and explanations of concepts are outstanding throughout.
- The methodology is novel to my knowledge and builds on recent progress in conformal prediction and causal methods for continuous treatments.
- Experimental settings and baseline methods are appropriate.
- The proposed method consistently achieves better empirical coverage than comparator methods.
summary: The authors propose a conformal prediction-based method for estimating uncertainty in the dose-response function, which defines the effect of continuous treatment on a continuous outcome, in the presence of confounders. Their method uses weighted conformal prediction, with weights based on generalized propensity scores. The presentation is exemplary and instructive throughout, including the motivation for and description of the proposed method. Experiments cover two established simulation settings and one new simulation setting. Results show that the resulting prediction intervals tend to be conservative, in the sense that empirical coverage of the true dose-response function is higher than intended.
weaknesses: - The evaluation is somewhat limited and focused almost entirely on empirical coverage.
- Error of the estimated CADRF is not presented except indirectly in Figure 2 for only one of the settings (Setup 3, Scenario 1).
- Empirical coverage is higher than desired in most cases and often very close to 1, and the prediction intervals are only shown for a single example.
- All this taken together makes me suspect that the method often yields excessively wide prediction intervals that may not be useful.
- The authors discuss the fact that the method yields conservative prediction intervals and provide brief explanations, but I think more discussion should be devoted to this given its central importance.
- I also think it is critical to provide figures akin to Figure 2 for more of the settings and compare error of the estimated CADRF between methods.

---

confidence: 2
contribution: 2
initial_rating: 5
presentation: 3
questions: How does the method perform with real-world data? It will make the method become more impactful and convincing with real data application analysis. I understand that applying real data for treatment effect estimation can be challenging, especially for continuous dose scenario. However, I encourage the authors to suggest specific real-world applications related to optimal dose recommendation, as this is an area where their method could provide significant insights. Probably some real data application deal with optimal dose level recommendation and use offline reward/value/outcome function to evaluate the performance of the estimated decision rule?
soundness: 2
strengths: The paper introduces a novel approach using conformal prediction to uncertainty quantification in dose-response models. The use of weighted conformal prediction ensures coverage across all treatment values, even under covariate shifts. The methodology has practical implications for personalized healthcare, drug dosing, and other fields requiring individualized treatment decisions. ​
summary: The paper introduces a new methodology for uncertainty quantification in dose-response models with continuous treatments using conformal prediction. ​ The approach leverages weighted conformal prediction, incorporating propensity estimation and kernel functions to address covariate shifts, ensuring coverage across all treatment values. ​ Building on the potential outcomes framework and generalized propensity scores, the method addresses some limitations in existing UQ techniques. ​ Experiments with synthetic data demonstrate its effectiveness, showing reliable prediction intervals with low treatment overlap. The practical implementation of this method can improve personalized dosing and interventions in various fields, enhancing decision-making by providing robust uncertainty quantification. ​
weaknesses: 1. The accuracy of the method relies heavily on the quality of the propensity score estimation, which can be challenging in real-world scenarios. In Section 5.2, the paper discussed using both oracle and estimated propensity distributions. How robust are their results to potential errors or biases in propensity score estimation? A sensitivity analysis could provide insights into how variations in the quality of propensity score estimation impact the overall accuracy of their method.

2. The experiments are conducted on synthetic data, and the method's performance in real-world applications remains to be fully validated. ​

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 2
questions: 1. In the method, they mentioned they use Conformal Prediction System (CPS), however, I do not see it in the Method section. Only in the numerical experiments, they mention it.However, it is not clear how they use it.

2. The numerical experiments are confusing to me. They consider eight different methods for comparison, but it is not clear to me which methods are their proposed methods. What is the difference between these methods, such as WCP local and WCP global?


3. I think covariate shift is a very common issue in causal inference, why the authors assume there is no distributional shift between the training and test data in terms of $\mathbf{X}$?

4. Is the uniform distribution assumption for the interventional distribution realistic?
soundness: 1
strengths: 1. The authors address a crucial issue in causal inference: estimating potential outcomes under continuous treatment. They also aim to provide a local guarantee for their proposed method, which is highly important in practical applications.

2. They have good literature review and make readers understand the background of the problem easily.

3. The method is relatively simple and easy to implement.
summary: In the manuscript, the authors propose a conformal prediction based method to obtain the interval estimation of the potential outcomes under continuous treatment. 
To achieve this, the authors use the weighted conformal prediction method. 
They also aim to provide a local guarantee for the proposed method via using the kernel weighting function.
weaknesses: 1. While the authors provide a method, they can not provide a theoretical guarantee for the proposed method. This is a significant drawback of the paper.

2. In my opinion, they do not illustrate the method well. The paper would benefit from more detailed illustrations for example an Algorithm or a flowchart.

3. The method relies on in my opinion a strong assumptions, that is interventional distribution is Uniform and there is not distributional shift between the training and test data in terms of $\mathbf{X}$.

4. The numerical experiments are not comprehensive enough and no real data application is provided.

---

confidence: 4
contribution: 2
initial_rating: 8
presentation: 2
questions: 1. How does the method perform when applied to real-world dose-response data, particularly in scenarios where confounding factors are not as easily modeled as in synthetic datasets?
2. Can the proposed method scale to larger datasets with higher-dimensional covariates and continuous treatments without a significant increase in computational time?
3. How robust is the propensity estimation in cases where the true propensity distribution is unknown or difficult to estimate? What are the limitations when using kernel density estimation (KDE) in practice?
4. Beyond healthcare, what other domains have been considered for the application of this method, and how would the assumptions about covariate shift differ in these contexts?
soundness: 2
strengths: 1. he paper presents an original application of conformal prediction to continuous treatment dose-response models, addressing an important gap in causal inference research. The integration of propensity score weighting and kernel-based adjustments to conformal prediction is a creative approach to ensure coverage under covariate shifts.
2. The paper is mostly clear, with well-structured sections that logically progress through the problem, related work, methodology, and experiments. The use of figures and visualizations to depict coverage is helpful for interpreting the results.
3. The problem of providing reliable prediction intervals for dose-response models has practical implications in many fields, such as personalized medicine, and this work represents a step forward in providing UQ in such contexts.
summary: This paper introduces a novel methodology for conformal prediction (CP) in dose-response models with continuous treatments, aiming to provide uncertainty quantification (UQ) for individualized decision-making. The approach leverages propensity score estimation and weighted conformal predictive systems to generate prediction intervals across a continuous range of treatments, which is essential for personalized healthcare and other decision-critical fields. By incorporating covariate shift assumptions and using kernel-based weighting, the authors propose a robust solution for achieving local coverage of dose-response predictions. The paper is validated on synthetic datasets, demonstrating the effectiveness of the proposed method.
weaknesses: 1. While the application is new, much of the methodology builds on existing CP and propensity score techniques without introducing fundamentally new theoretical contributions. The added value lies in the application context, but more could be done to differentiate this work from prior studies.
2. The reliance on synthetic datasets raises concerns about the method's practical utility. A more thorough evaluation on real-world data would strengthen the paper’s claim of addressing practical challenges in dose-response modeling.
3. Although the authors mention the efficiency improvements from weighted conformal prediction, the scalability of the method, particularly in real-time applications, remains unclear. Detailed analysis of the computational overhead, especially with large-scale data, would be beneficial.

---

confidence: 3
contribution: 1
initial_rating: 5
presentation: 2
questions: Clarifying clear differences to prior similar works,
Convincing validation
soundness: 3
strengths: This paper targets a significant and challenging task of considering uncertainty in CATE estimation when treatment is continuous.
summary: This paper addresses continuous treatment’s CATE via weighted conformal prediction.
weaknesses: -	Methodological contribution compared to prior work is incremental. The proposed idea of estimating counterfactual outcome interval using weighted conformal prediction has already been published by Lei et al. Lei et al. have proven that a generalized propensity score can be used for the weight in conformal prediction. The main difference between this paper and the similar work by Lei et al. is estimation targets (continuous CATE in this paper vs. discrete CATE in prior work). 
-	Compared to another prior work by Schroder et al., this paper’s methodological contribution is also marginal. The discussion in Supplement C is not fully convincing in distinguishing this paper’s contribution from the prior work.  
-	Novelty is limited. This paper applies an existing method to an existing task. No new approach or new generalizable insight was provided.  
-	Validation is limited. Neither a formal theoretical guarantee nor empirical validation with real data were provided. I understand the lack of ground truth in the CATE world, but I would appreciate it if a theoretical guarantee could supplement the synthetic data validation. No comparison to baseline models.
-	Therefore, this paper does not have a broader impact on the following works in this field.


Reference

Lihua Lei, Emmanuel J. Candès, Conformal Inference of Counterfactuals and Individual Treatment Effects, Journal of the Royal Statistical Society Series B: Statistical Methodology, Volume 83, Issue 5, November 2021, Pages 911–938, https://doi.org/10.1111/rssb.12445

Meta-review: This paper proposed a conformal prediction-based method for uncertainty quantification in dose-response models with continuous treatments. The approach leverages weighted conformal prediction, incorporating propensity estimation and kernel functions to address covariate shifts. The problem is well motivated and the extension of conformal inference to continuous treatments is important. The initial reviews mostly questioned on the novelty and contribution of this work. Neither a formal theoretical guarantee nor empirical validation with real data were provided. While a great effort has been made by authors during rebuttal to address these issues, the main concerns remain. The proposed method is a direct extension or integration of conformal prediction and propensity score techniques without introducing fundamentally new theoretical contributions. Both the theoretical and methodological innovations are limited.  

In any case, this is clearly a borderline paper. It is interesting but also has a low originality and weak significance. For that reason I think it is not ready for ICLR. We'd encourage the authors to take into consideration all the feedback provided by the reviewers to strengthen their manuscript for resubmission.

Decision: Reject