Paper: Bayesian Preference Elicitation for Personalized Prefactual Recommendation (ICLR-2024)

Reviewer comment (Experiments):
The paper fails to discuss and compare some important baselines, which diminishes the novelty and soundness of the approach. Specifically, [1] offers personalized recourse options by learning a cost function from experts, which would serve as a strong baseline to start with.

Author response:
Thank you for highlighting the need for more extensive baseline comparisons. We acknowledge that the AReS model from [1], which focuses on global counterfactual explanations for wide-ranging population segments, offers a different perspective with its interpretable recourse summaries. Recognizing its relevance, we plan to include this model in our related work section and use it as a baseline in our experiments. This approach will help in generating a more informed prior for question determination, addressing the cold-start issue effectively. Regarding the distinctions between our work and [2], we have provided a detailed explanation in our response to the following Q2.

Reviewer comment (Novelty):
[2] provides a Bayesian framework for learning personalized cost functions for algorithmic recourse via pairwise queries, which is robust to user uncertainty. This is not discussed in the paper despite being relevant.

Author response:
Thank you for your inquiry. We have summarized the difference between our work and [2] as follows:

* **Difference in personalized cost function:** Unlike [2], which utilizes a linear Structural Causal Model with parameterized weight for the cost function, we employ the Mahalanobis distance to model our cost function. The reasons for choosing Mahalanobis distance and its advantages over the $\ell_1$ distance are detailed in Section 2.
* **Difference in problem determination:** The approach in [2] involves greedy optimization to identify an informative choice set that maximizes the Expected Utility Score (EUS), akin to listwise comparison. In contrast, our paper primarily addresses pairwise comparison, with an extension to listwise comparison discussed in Appendix A. Our goal is to identify item pairs that maximize the mutual information with the preference matrix, considering its prior distribution. We also discuss the computational benefits of pairwise over listwise comparison in Appendix A.
* **Difference in posterior update:** Contrary to [2]'s Bayesian inference method for inferring the posterior over weights, our method updates the posterior over the preference matrix by balancing prior-posterior distortion minimization and likelihood maximization. This approach enhances the robustness of our framework, especially in the face of user response inconsistencies.
* **Difference in recourse generation:** Our recourse generation method is graph-based, following the FACE method, as opposed to [2]'s reliance on a reinforcement learning-based approach, W-FARE.

The outlined distinctions highlight the unique contributions and innovative aspects of our Bayesian PR method in contrast to the methodology employed in [2]. To provide a more thorough understanding, we will incorporate this comparative analysis into both the discussion of related work and the experimental sections of our paper.

We conduct a performance comparison involving Bayesian PR, PEAR [2], and FACE on the Adult dataset. For each method, we generate the user's cost matrix $A_0$ as detailed in Section 5. Subsequently, we report the average cost and deviation across 10 users for two scenarios: using the Mahalanobis distance and the $\ell_1$ norm as the true cost functions, respectively. Notably, our Bayesian PR method achieves the lowest cost among the evaluated methods.

Table: Comparison of Bayesian PR, PEAR, and FACE using Mahalanobis distance and $\ell_1$ norm as the cost

| Method | Mahalanobis distance | $\ell_1$ norm |
| --- | --- | --- |
| Bayesian PR | **1.16 $\pm$ 0.67** | **1.45 $\pm$ 1.07** |
| PEAR | 1.22 $\pm$ 0.59 | 1.84 $\pm$ 1.25 |
| FACE | 1.56 $\pm$ 0.29 | 1.68 $\pm$ 0.54 |

Reviewer comment (Novelty):
It is not clear what is the advantage or originality of the proposed Bayesian PR over the method in [2], which is cited in the paper but not discussed anywhere in the main manuscript.

Author response:
Thank you for your inquiry. We have summarized the difference between our work and [2] as follows:

* **Difference in personalized cost function:** Unlike [2], which utilizes a linear Structural Causal Model with parameterized weight for the cost function, we employ the Mahalanobis distance to model our cost function. The reasons for choosing Mahalanobis distance and its advantages over the $\ell_1$ distance are detailed in Section 2.
* **Difference in problem determination:** The approach in [2] involves greedy optimization to identify an informative choice set that maximizes the Expected Utility Score (EUS), akin to listwise comparison. In contrast, our paper primarily addresses pairwise comparison, with an extension to listwise comparison discussed in Appendix A. Our goal is to identify item pairs that maximize the mutual information with the preference matrix, considering its prior distribution. We also discuss the computational benefits of pairwise over listwise comparison in Appendix A.
* **Difference in posterior update:** Contrary to [2]'s Bayesian inference method for inferring the posterior over weights, our method updates the posterior over the preference matrix by balancing prior-posterior distortion minimization and likelihood maximization. This approach enhances the robustness of our framework, especially in the face of user response inconsistencies.
* **Difference in recourse generation:** Our recourse generation method is graph-based, following the FACE method, as opposed to [2]'s reliance on a reinforcement learning-based approach, W-FARE.

The outlined distinctions highlight the unique contributions and innovative aspects of our Bayesian PR method in contrast to the methodology employed in [2]. To provide a more thorough understanding, we will incorporate this comparative analysis into both the discussion of related work and the experimental sections of our paper.

We conduct a performance comparison involving Bayesian PR, PEAR [2], and FACE on the Adult dataset. For each method, we generate the user's cost matrix $A_0$ as detailed in Section 5. Subsequently, we report the average cost and deviation across 10 users for two scenarios: using the Mahalanobis distance and the $\ell_1$ norm as the true cost functions, respectively. Notably, our Bayesian PR method achieves the lowest cost among the evaluated methods.

Table: Comparison of Bayesian PR, PEAR, and FACE using Mahalanobis distance and $\ell_1$ norm as the cost

| Method | Mahalanobis distance | $\ell_1$ norm |
| --- | --- | --- |
| Bayesian PR | **1.16 $\pm$ 0.67** | **1.45 $\pm$ 1.07** |
| PEAR | 1.22 $\pm$ 0.59 | 1.84 $\pm$ 1.25 |
| FACE | 1.56 $\pm$ 0.29 | 1.68 $\pm$ 0.54 |

Reviewer comment (Experiments):
The experimental section does not include [1] and [2] as baselines, even though they are closely related and should be considered for testing.

Author response:
Thank you for highlighting the need for more extensive baseline comparisons. We acknowledge that the AReS model from [1], which focuses on global counterfactual explanations for wide-ranging population segments, offers a different perspective with its interpretable recourse summaries. Recognizing its relevance, we plan to include this model in our related work section and use it as a baseline in our experiments. This approach will help in generating a more informed prior for question determination, addressing the cold-start issue effectively. Regarding the distinctions between our work and [2], we have provided a detailed explanation in our response to the following Q2.  

Thank you for your inquiry. We have summarized the difference between our work and [2] as follows:

* **Difference in personalized cost function:** Unlike [2], which utilizes a linear Structural Causal Model with parameterized weight for the cost function, we employ the Mahalanobis distance to model our cost function. The reasons for choosing Mahalanobis distance and its advantages over the $\ell_1$ distance are detailed in Section 2.
* **Difference in problem determination:** The approach in [2] involves greedy optimization to identify an informative choice set that maximizes the Expected Utility Score (EUS), akin to listwise comparison. In contrast, our paper primarily addresses pairwise comparison, with an extension to listwise comparison discussed in Appendix A. Our goal is to identify item pairs that maximize the mutual information with the preference matrix, considering its prior distribution. We also discuss the computational benefits of pairwise over listwise comparison in Appendix A.
* **Difference in posterior update:** Contrary to [2]'s Bayesian inference method for inferring the posterior over weights, our method updates the posterior over the preference matrix by balancing prior-posterior distortion minimization and likelihood maximization. This approach enhances the robustness of our framework, especially in the face of user response inconsistencies.
* **Difference in recourse generation:** Our recourse generation method is graph-based, following the FACE method, as opposed to [2]'s reliance on a reinforcement learning-based approach, W-FARE.

The outlined distinctions highlight the unique contributions and innovative aspects of our Bayesian PR method in contrast to the methodology employed in [2]. To provide a more thorough understanding, we will incorporate this comparative analysis into both the discussion of related work and the experimental sections of our paper.

We conduct a performance comparison involving Bayesian PR, PEAR [2], and FACE on the Adult dataset. For each method, we generate the user's cost matrix $A_0$ as detailed in Section 5. Subsequently, we report the average cost and deviation across 10 users for two scenarios: using the Mahalanobis distance and the $\ell_1$ norm as the true cost functions, respectively. Notably, our Bayesian PR method achieves the lowest cost among the evaluated methods.

Table: Comparison of Bayesian PR, PEAR, and FACE using Mahalanobis distance and $\ell_1$ norm as the cost

| Method | Mahalanobis distance | $\ell_1$ norm |
| --- | --- | --- |
| Bayesian PR | **1.16 $\pm$ 0.67** | **1.45 $\pm$ 1.07** |
| PEAR | 1.22 $\pm$ 0.59 | 1.84 $\pm$ 1.25 |
| FACE | 1.56 $\pm$ 0.29 | 1.68 $\pm$ 0.54 |

Reviewer comment (Novelty):
What is the difference between the proposed Bayesian PR and the method proposed in [2]? After a closer look, they appear closely related, and clarification on how the proposed approach differs from the one presented in [2] would be appreciated.

Author response:
Thank you for your inquiry. We have summarized the difference between our work and [2] as follows:

* **Difference in personalized cost function:** Unlike [2], which utilizes a linear Structural Causal Model with parameterized weight for the cost function, we employ the Mahalanobis distance to model our cost function. The reasons for choosing Mahalanobis distance and its advantages over the $\ell_1$ distance are detailed in Section 2.
* **Difference in problem determination:** The approach in [2] involves greedy optimization to identify an informative choice set that maximizes the Expected Utility Score (EUS), akin to listwise comparison. In contrast, our paper primarily addresses pairwise comparison, with an extension to listwise comparison discussed in Appendix A. Our goal is to identify item pairs that maximize the mutual information with the preference matrix, considering its prior distribution. We also discuss the computational benefits of pairwise over listwise comparison in Appendix A.
* **Difference in posterior update:** Contrary to [2]'s Bayesian inference method for inferring the posterior over weights, our method updates the posterior over the preference matrix by balancing prior-posterior distortion minimization and likelihood maximization. This approach enhances the robustness of our framework, especially in the face of user response inconsistencies.
* **Difference in recourse generation:** Our recourse generation method is graph-based, following the FACE method, as opposed to [2]'s reliance on a reinforcement learning-based approach, W-FARE.

The outlined distinctions highlight the unique contributions and innovative aspects of our Bayesian PR method in contrast to the methodology employed in [2]. To provide a more thorough understanding, we will incorporate this comparative analysis into both the discussion of related work and the experimental sections of our paper.

We conduct a performance comparison involving Bayesian PR, PEAR [2], and FACE on the Adult dataset. For each method, we generate the user's cost matrix $A_0$ as detailed in Section 5. Subsequently, we report the average cost and deviation across 10 users for two scenarios: using the Mahalanobis distance and the $\ell_1$ norm as the true cost functions, respectively. Notably, our Bayesian PR method achieves the lowest cost among the evaluated methods.

Table: Comparison of Bayesian PR, PEAR, and FACE using Mahalanobis distance and $\ell_1$ norm as the cost

| Method | Mahalanobis distance | $\ell_1$ norm |
| --- | --- | --- |
| Bayesian PR | **1.16 $\pm$ 0.67** | **1.45 $\pm$ 1.07** |
| PEAR | 1.22 $\pm$ 0.59 | 1.84 $\pm$ 1.25 |
| FACE | 1.56 $\pm$ 0.29 | 1.68 $\pm$ 0.54 |

Reviewer comment (Experiments):
Missing comparison to sampling and evaluation of approximation quality.

Author response:
**Q4:** How does sampling compare to the analytical solution? A comparison to sampling is missing from the experiments. This is important for validating the quality of the approximate analytical solution as well as the computational efficiency claims. \
**Response:** Thank you for highlighting this aspect. Our discussion following Theorem 3.2 outlines the computational efficiency of the analytical approach over sampling, particularly in terms of time complexity. Regarding the validation of the approximation's quality, we have presented histograms in the experimental section that illustrate Mahalanobis distances for various $\kappa$ values ($+\infty$, 1, 5, 10) at $T=10$. These findings indicate a clear trend: as $T$ increases, the means of these distributions consistently align closer to the actual cost values for all $\kappa$ values, with a corresponding reduction in standard deviations. Acknowledging the importance of your suggestion, we incorporate a comparison with sampling in future revisions, focusing on both the quality of the results and execution time.

We test the estimated probability and the analytic probability on four different dimensionalities. For each dimensionality, we perform Monte Carlo sampling with 100000 samples and evaluate $\langle A_i, M \rangle \leq 0, ~ i = 1, \cdots, 100000$. We report the computation time for both approaches. As we can see in the following table, the estimated probabilities match the analytical probability which concludes the validity of the analytic form. Furthermore, the computation times of the estimated probabilities grow more significantly than the analytic method as we deal with higher dimensions, justifying the need to adopt analytic probability. 

Table: Comparison with sampling method for time and probability value
| dimensionality | estimated-probability | estimated-time | analytic-probability | analytic-time |
| --- | --- | --- | --- | --- |
| 2   | 0.0020 | 1.3270 sec | 0.0019 | 0.0009 sec |
| 5   | 0.0415 | 1.5225 sec | 0.0417 | 0.0022 sec |
| 10  | 0.0099 | 2.7120 sec | 0.0098 | 0.0436 sec |
| 15  | 0.9915 | 3.6238 sec | 0.9915 | 0.0516 sec |

Reviewer comment (Theory):
Theorem 3.2 assumes that kappa goes to infinity which corresponds to noiseless responses. How to understand Eq (3) (and the rest of section 4) with kappa = inf?

Author response:
**Q1:**  How to understand Eq (3) (and the rest of section 4) with kappa=inf? Also, the link function is approximated by a linear function (section 4.1), so how would that work when kappa=inf?\
**Response:** Thank you for your question. When considering $\kappa=\infty$, as outlined in the proof of Proposition 1 and by applying the Dominated Convergence Theorem, we derive the following limit:
\begin{equation*}
\lim_ {\kappa\rightarrow \infty} \text{Prob}(\tilde R_{ij}^\kappa(\tilde A)\neq R_{ij}) = \int_ {S_+^d} \lim_ {\kappa \rightarrow \infty} \Phi(\kappa R_{ij}\Delta_{ij}(S))f_P(S) dS = \int_ {S_+^d} \mathrm{1}_ {\{R_{ij}\Delta_{ij}(S)\geq 0\}} f_P(S)dS.
\end{equation*}
In this scenario, we can also use the linear function $v \mapsto v$ to approximate the step function $v \mapsto \mathrm{1}_ {\{v \geq 0\}}$. This approximation leads us to the following expression when $\kappa = \infty$
\begin{equation*}
\text{Prob}(\tilde R_{ij}^\kappa(\tilde A)\neq R_{ij}) \approx E_P[R_{ij}\langle M_{ij}, \tilde A\rangle].
\end{equation*}
Accordingly, we can set $\tau=1$ in this context. We will include the discussion of the case $\kappa=\infty$ in the Appendix.

Reviewer comment (Experiments):
It may be good to show empirically how the assumption kappa → ∞ affects the results versus using the final kappa.

Author response:
**Q2:**  May be good to show empirically how this assumption affects the results vs using final kappa. \
**Response:** We have empirically tested the impact of the $\kappa$ assumption, showcased in Figures 4 to 7 in Appendix D. These histograms display Mahalanobis distances for different $\kappa$ values ($+\infty$, 1, 5, 10) at $T=10$. The results reveal a consistent trend: with increasing $T$, the distribution means converge towards the true cost values for all $\kappa$ values, accompanied by reduced standard deviations. This indicates our framework's robustness and precision, effectively handling variations in finite $\kappa$.

Reviewer comment (Presentation):
There are several layers of approximation (e.g., kappa = inf, approximating sigmoid with a linear function, finite iterations for the projection in Alg 1). It would be helpful to clearly state all of them in a single place before giving all the details.

Author response:
**Q3:** There are several layers of approximation, it would be helpful to clearly state all of them in a single place before giving all the details. It may also be helpful to do some ablation studies and see how each level of approximation affects the overall performance. \
**Response:** Thank you for the suggestion. Our method primarily involves two approximations: first, we consider the case $\kappa = \infty$ in mutual information computation for an analytical expression (Section 3); second, we approximate the sigmoid function with a linear function for posterior updates. The computational error in Algorithm 1, controlled by the stopping criteria $\\|\Sigma_{(k)}-\Sigma_{(k-1)}\\|<10^{-6}$, is minimal. Empirical results (Figures 4 to 7) show that as iterations increase, the approximations yield values closer to the true costs for all datasets and $\kappa$ values, with diminishing standard deviations. Detailed ablation studies for each approximation level remain a potential future research direction.

Reviewer comment (Experiments):
It may be helpful to do some ablation studies and see how each level of approximation affects the overall performance.

Author response:
**Q3:** There are several layers of approximation, it would be helpful to clearly state all of them in a single place before giving all the details. It may also be helpful to do some ablation studies and see how each level of approximation affects the overall performance. \
**Response:** Thank you for the suggestion. Our method primarily involves two approximations: first, we consider the case $\kappa = \infty$ in mutual information computation for an analytical expression (Section 3); second, we approximate the sigmoid function with a linear function for posterior updates. The computational error in Algorithm 1, controlled by the stopping criteria $\\|\Sigma_{(k)}-\Sigma_{(k-1)}\\|<10^{-6}$, is minimal. Empirical results (Figures 4 to 7) show that as iterations increase, the approximations yield values closer to the true costs for all datasets and $\kappa$ values, with diminishing standard deviations. Detailed ablation studies for each approximation level remain a potential future research direction.

Reviewer comment (Experiments):
A comparison to sampling is missing from the experiments. This is important for validating the quality of the approximate analytical solution as well as the computational efficiency claims.

Author response:
**Q4:** How does sampling compare to the analytical solution? A comparison to sampling is missing from the experiments. This is important for validating the quality of the approximate analytical solution as well as the computational efficiency claims. \
**Response:** Thank you for highlighting this aspect. Our discussion following Theorem 3.2 outlines the computational efficiency of the analytical approach over sampling, particularly in terms of time complexity. Regarding the validation of the approximation's quality, we have presented histograms in the experimental section that illustrate Mahalanobis distances for various $\kappa$ values ($+\infty$, 1, 5, 10) at $T=10$. These findings indicate a clear trend: as $T$ increases, the means of these distributions consistently align closer to the actual cost values for all $\kappa$ values, with a corresponding reduction in standard deviations. Acknowledging the importance of your suggestion, we incorporate a comparison with sampling in future revisions, focusing on both the quality of the results and execution time.

We test the estimated probability and the analytic probability on four different dimensionalities. For each dimensionality, we perform Monte Carlo sampling with 100000 samples and evaluate $\langle A_i, M \rangle \leq 0, ~ i = 1, \cdots, 100000$. We report the computation time for both approaches. As we can see in the following table, the estimated probabilities match the analytical probability which concludes the validity of the analytic form. Furthermore, the computation times of the estimated probabilities grow more significantly than the analytic method as we deal with higher dimensions, justifying the need to adopt analytic probability. 

Table: Comparison with sampling method for time and probability value
| dimensionality | estimated-probability | estimated-time | analytic-probability | analytic-time |
| --- | --- | --- | --- | --- |
| 2   | 0.0020 | 1.3270 sec | 0.0019 | 0.0009 sec |
| 5   | 0.0415 | 1.5225 sec | 0.0417 | 0.0022 sec |
| 10  | 0.0099 | 2.7120 sec | 0.0098 | 0.0436 sec |
| 15  | 0.9915 | 3.6238 sec | 0.9915 | 0.0516 sec |

Reviewer comment (Theory):
The graph-based approach of section 5 considers only transitions between points from the training data, but it may be possible to transition to new points not in the data (possibly unrealizable). Can such transitions be accommodated in your framework? If we have access to the classifier we can know the label of any point.

Author response:
**Q7:** The graph-based approach of section 5 considers only transitions between points from the training data, but it may be possible to transition to new points not in the data (possibly unrealizable). Can such transitions be accommodated in your framework? If we have access to the classifier we can know the label of any point. \
**Response:** Yes. Our framework can theoretically accommodate transitions to new data points not present in the training data. This would involve enumerating these new points and integrating them alongside the training data within our graph structure. However, this approach raises concerns about the actionability of such transitions, as these new data points might be unrealizable in practical scenarios. In line with maintaining actionable recourse strategies, we adhere to the FACE methodology, which constructs the graph using only training data. This decision ensures that the generated recourses are both realistic and feasible.

Reviewer comment (Reproducibility):
What are typical path lengths in the experiments?

Author response:
**Q8:** What are typical path lengths in the experiments? \
**Response:** In our experiments, the average path lengths observed for each dataset were as follows: Synthesis: 2.07, German: 2.54, Bank: 5.47, Student: 5.31.

Reviewer comment (Writing):
What is the dimension d in section 6.1?

Author response:
**Q9:** What is the dimension $d$? \
**Response:** The dimension $d$ represents the dimension of $x$.

Reviewer comment (Experiments):
The model in section 6.1 seems rather small (90 neurons in 3 MLP layers).

Author response:
**Q10:** The model seems rather small (90 neurons in 3 MLP layers).\
**Response:**  For the classifier, we adhere to the setup used by Wachter et al. (2017). The classifier's accuracy details are included in Appendix D of our updated paper.

Reviewer comment (Experiments):
What is the justification for the choice τκ = 1 in section 6.1?

Author response:
**Q11:** What is the justification for the choice $\tau\kappa = 1$?\
**Response:** Regarding the choice of $\tau\kappa = 1$, this is based on the inverse relationship between $\tau$ and $\kappa$: by setting $\tau$ as $1/\kappa$, we ensure their product equals 1. While alternative values for $\tau$ could be determined through cross-validation, this process can be time-intensive. Importantly, our empirical findings demonstrate that the choice of $\tau \kappa=1$ yields robust performance. Specifically, this parameter setting results in lower costs when compared to other baselines, thereby validating its efficacy within the framework of our model.

Reviewer comment (Evaluation):
No significant improvement in comparison with the baseline.

Author response:
We apologize for any misunderstanding. In the context of Table 1, we specifically use Mahalanobis distance as the true cost. In this setup, our framework accurately specifies the cost function type to align with the Mahalanobis distance, whereas FACE misspecifies it. As a result, our method demonstrates improved cost efficiency compared to FACE when Mahalanobis distance is the true cost.

Turning to Table 2, where the true cost is modeled as the $\ell_1$ norm, FACE specifies the cost function correctly, while our framework does not. Nonetheless, even with this misspecification, our method exhibits competitive performance, matching or surpassing FACE in cost efficiency on the Synthetic and Student datasets. This contrast between Tables 1 and 2 underscores the adaptability and resilience of our approach in handling various cost function scenarios.

Reviewer comment (Theory):
It's not clear enough why this paper applies Mahalanobis distance. Why does this distance make sense to serve as the cost function? How is it compared with many other distance functions and other cost functions?

Author response:
In Section 2, we justify our choice of Mahalanobis distance over Euclidean or Manhattan ($\ell_1$) distances for preference modeling. This distance stands out for its ability to account for variances and correlations among dimensions, providing a nuanced view of individual, non-uniform preferences and subjective 'closeness', which Euclidean distance, treating all dimensions equally, fails to capture. Additionally, Mahalanobis distance incorporates a personalized matrix $A$, relevant in scenarios where costs involve cognitive elements and complex preference structures. This metric's effectiveness is widely recognized in the field of metric learning, as evidenced by literature like Kulis (2013), Yang \& Jin (2006), and Bellet et al. (2013). Its adaptability to unique data distributions makes it particularly suitable for intricate, subjective decision-making modeling. 

Furthermore, minimizing this distance aligns with the innate human tendency to achieve desired outcomes with the least effort. By using the Mahalanobis distance, we create a Riemannian manifold, introducing an element of curvature through the preference matrix $A$ [A]. This approach effectively encapsulates the complex interplay between preference and cost, enabling us to accurately model the diverse aspects of individual decision-making.


[A] Arvanitidis, Georgios, Lars K. Hansen, and S$ø$ren Hauberg. "A locally adaptive normal distribution." Advances in Neural Information Processing Systems 29 (2016).

Reviewer comment (Theory):
The complexity of the proposed algorithm is unclear and there is no analysis on how many samples (how large \(M\)) are needed for the algorithm.

Author response:
As for the complexity of the proposed algorithm, please refer to the response to Q1 in the global response section. Regarding the number of samples ($M$), our algorithm doesn't impose a specific limit. The choice of $M$ affects performance and computational efficiency: larger $M$ enhances robustness but increases computation, while smaller $M$ reduces computational load but may affect outcomes.

Reviewer comment (Evaluation):
The proposed algorithm does not appear to be better than the baselines because it has almost the same cost as FACE if using \(\ell_1\) as the true cost. The improvement in Mahalanobis distance as the true cost is not compelling because this is the same distance used in the algorithm and, to some extent, the algorithm cracks the ground truth.

Author response:
We apologize for any misunderstanding. In the context of Table 1, we specifically use Mahalanobis distance as the true cost. In this setup, our framework accurately specifies the cost function type to align with the Mahalanobis distance, whereas FACE misspecifies it. As a result, our method demonstrates improved cost efficiency compared to FACE when Mahalanobis distance is the true cost.

Turning to Table 2, where the true cost is modeled as the $\ell_1$ norm, FACE specifies the cost function correctly, while our framework does not. Nonetheless, even with this misspecification, our method exhibits competitive performance, matching or surpassing FACE in cost efficiency on the Synthetic and Student datasets. This contrast between Tables 1 and 2 underscores the adaptability and resilience of our approach in handling various cost function scenarios.

Reviewer comment (Experiments):
The baselines are not compared in Section 6.3.

Author response:
In Section 6.3, we focused on comparing our method with the graph-based FACE method due to its direct relevance. Comparisons with non-graph-based methods are detailed in the Appendix, owing to space constraints in the main text, ensuring a comprehensive yet concise presentation.

Reviewer comment (Theory):
The complexity analysis of the whole proposed algorithm is missing or lacks discussion.

Author response:
Please refer to the response to Q1 in the global response part.