Paper: Escaping the Sample Trap: Fast and Accurate Epistemic Uncertainty Estimation with Pairwise-Distance Estimators (ICLR-2024)

Reviewer comment (Experiments):
Only NFLows Base is considered as the base learner method for ensemble learning. It would be nice to see if the proposed estimator can also be applied to other scenarios.

Author response:
Additional Base Learners:  
   
We do consider Probabilistic Network Ensembles as an additional base learner in the Appendix.

Reviewer comment (Theory):
eq 1, needs to specify pi_j to be between 0 and 1.

Author response:
Writing Suggestions:  
   
Thank you, these have been implemented and are reflected in the new version of the paper.

Reviewer comment (Writing):
Grammar: "in addition to the improving the lower bound".

Author response:
Writing Suggestions:  
   
Thank you, these have been implemented and are reflected in the new version of the paper.

Reviewer comment (Presentation):
Would be nice to list the environment dimension and other related statistics in Table 1 for readability.

Author response:
Writing Suggestions:  
   
Thank you, these have been implemented and are reflected in the new version of the paper.

Reviewer comment (Experiments):
The baseline-algorithms are not sufficient; Random (query), BADGE (Ash et al., 2019) and BatchBALD (Kirsch et al., 2019) can be other base-line algorithms.

Author response:
Baselines:
 
Random query is used as a baseline, as detailed in Table 1-5 and depicted in Figures 7, 8, 10, and 13. Our primary contribution lies in presenting a more precise estimator of epistemic uncertainty in higher dimensions. Consequently, we have adopted BALD (Houlsby et al. 2011) as a baseline in our experiments. BADGE (Ash et al., 2019) computes the gradient of the last layer outputs and employs k-means++ on these gradients to select data points for acquisition. In their original implementation, they exclusively focused on classification tasks with gradients of size 256x10. In our experiments, we face the challenge of computing gradients of size 256x257x2 and subsequently applying k-means++. However, k-means++ encounters difficulties in tasks with such elevated dimensionality. Moreover, BatchBALD necessitates the estimation of $p(y_1,...,y_n)$, a task feasible in classification but poses a barrier in high-dimensional regression output where $y_i$ represents a continuous output in a high-dimensional space.

Reviewer comment (Experiments):
The author(s) claim that the proposed algorithm can be strong for the large size of queries; therefore, the reviewer wants to see the compared results with algorithms considering the diversity.

Author response:
Baselines:
 
Random query is used as a baseline, as detailed in Table 1-5 and depicted in Figures 7, 8, 10, and 13. Our primary contribution lies in presenting a more precise estimator of epistemic uncertainty in higher dimensions. Consequently, we have adopted BALD (Houlsby et al. 2011) as a baseline in our experiments. BADGE (Ash et al., 2019) computes the gradient of the last layer outputs and employs k-means++ on these gradients to select data points for acquisition. In their original implementation, they exclusively focused on classification tasks with gradients of size 256x10. In our experiments, we face the challenge of computing gradients of size 256x257x2 and subsequently applying k-means++. However, k-means++ encounters difficulties in tasks with such elevated dimensionality. Moreover, BatchBALD necessitates the estimation of $p(y_1,...,y_n)$, a task feasible in classification but poses a barrier in high-dimensional regression output where $y_i$ represents a continuous output in a high-dimensional space.

Reviewer comment (Writing):
Q1: Is the notation of \( P_{b \mid b} \) in 31 lines on page 5 right?

Author response:
Q1:
 
Thank you for pointing this out, we have addressed this small typo in the text.

Reviewer comment (Writing):
Q2: Please clarify what is \( f(a_t, s_t) = s_{t+1} \) in a detailed manner.

Author response:
Q2:
 
The function $f$ denotes a transition function: given a state and action, denoted as $s_t$ and $a_t$ respectively, it produces the subsequent state $s_{t+1}$. In our scenario, these functions are intricate and nonlinear due to our consideration of robot dynamics. In the paper, we explicitly note this as a dynamics model, stating, "the dynamics model for each environment was modeled, $f_{\theta} (s_t, a_t) = s_{t+1}$."

Reviewer comment (Presentation):
Q3: Number of samples in Figure 3: what’s the meaning; probably, size of unlabeled samples at initial steps, right?

Author response:
Q3:
 
Monte Carlo (MC) estimators necessitate the drawing of a certain number of samples to provide accurate estimates; typically, fewer samples lead to poorer results. Figure 3 illustrates this phenomena within a specific environment. To enhance clarity, we have modified the text in the paper accordingly:
 
To conduct a more in-depth analysis of our proposed method, we compared PaiDEs to MC estimators with a varying sample size $per$ $x_i$ in the Hopper-v2 environment. We expected that MC estimators would perform on par with PaiDEs with a sufficient number of samples. However, as illustrated in Figure 3,MC estimators fell short of achieving the same level of performance as PaiDEs in this particular scenario. This suggests that, taking into consideration hardware constraints, PaiDEs begin to outperform MC estimators when dealing with 11 dimensional outputs.