Paper: High-Dimensional Safe Exploration via Optimistic Local Latent Safe Optimization (ICLR-2024)

Reviewer comment (Theory):
The assumptions limit the widespread application of the proposed algorithm.

Author response:
**The assumptions limit the widespread application.**

We argue that the most assumptions we used in the paper are common-used in high-dimensional BO literature. Assumptions 1 and 2 are regularity assumptions of the function, which enables to use Gaussian process to model function distributions. Assumption 3 is low effective dimension assumption common-used in embedding BO works to manifest the structural reward characteristic. Note the low-dimensional manifold are always observed in most high-dimensional structures. The only additional assumption is the distance preserving ability of the latent space embedding, which is crucial to guarantee probabilistic safety and can be satisfied using some linear embedding, or approximately satisfied using isometry regularized VAE. We have demonstrated success of our proposed method in two different complex high-dimensional problems and one real-world application. And we expect to apply OLLSO to more real-word safety-critical problems.

Reviewer comment (Presentation):
The presentation has the potential to be improved, especially Section 4.1.

Author response:
**The presentation has the potential to be improved and the contributions could be better summarized**

Thank you for your suggestion, and we will improve the presentation in the next version.

Here we summarize our contribution to as follows:

1. In this paper, we propose a practical method to achieve safe and efficient optimization of high-dimensional functions. To our best knowledge, no existing work focuses on guaranteeing safety during the high-dimensional optimization.

2. OLLSO overcomes the scalability and efficiency difficulties of the previous GP-based safe optimization method by introducing a local search mechanism with optimistic safety region identification, and achieves safe and efficient optimization of functions with hundreds of dimensions.

 3. OLLSO is the first algorithm to use distance-preserved VAE to address safety issue in latent space optimization. We derive the theoretical probabilistic safety guarantee of OLLSO, which enable it to handle extremely high-dimension or discrete input space.

 4. Our experimental results shows that OLLSO significantly outperforms existing safe BO methods, constrained BO algorithms, and direct combinations with dimension reduction models.

We want to emphasize that OLLSO is a PRACTICAL method that can safely optimize high-dimensional real-world problems. In addition to state-of-the-art simulation performance, our real clinical experiments show that OLLSO successfully optimizes neural stimulation for the control of human muscles. While the majority of our cited papers conduct their experiments only in simulation or synthetic problems, OLLSO is both practical and theoretically verifiable to safely and efficiently optimize real-world safety-critical problems.

Reviewer comment (Writing):
The contributions could be better summarized.

Author response:
**The presentation has the potential to be improved and the contributions could be better summarized**

Thank you for your suggestion, and we will improve the presentation in the next version.

Here we summarize our contribution to as follows:

1. In this paper, we propose a practical method to achieve safe and efficient optimization of high-dimensional functions. To our best knowledge, no existing work focuses on guaranteeing safety during the high-dimensional optimization.

2. OLLSO overcomes the scalability and efficiency difficulties of the previous GP-based safe optimization method by introducing a local search mechanism with optimistic safety region identification, and achieves safe and efficient optimization of functions with hundreds of dimensions.

 3. OLLSO is the first algorithm to use distance-preserved VAE to address safety issue in latent space optimization. We derive the theoretical probabilistic safety guarantee of OLLSO, which enable it to handle extremely high-dimension or discrete input space.

 4. Our experimental results shows that OLLSO significantly outperforms existing safe BO methods, constrained BO algorithms, and direct combinations with dimension reduction models.

We want to emphasize that OLLSO is a PRACTICAL method that can safely optimize high-dimensional real-world problems. In addition to state-of-the-art simulation performance, our real clinical experiments show that OLLSO successfully optimizes neural stimulation for the control of human muscles. While the majority of our cited papers conduct their experiments only in simulation or synthetic problems, OLLSO is both practical and theoretically verifiable to safely and efficiently optimize real-world safety-critical problems.

Reviewer comment (Presentation):
The presentation in Section 4 should be better polished. In my mind, the definitions of the acquisition function A and the local region are missed at least. It would also be helpful if the authors could make a plot to show what they (side length, local region...) are.

Author response:
**Methodology clarification.**

We describe the choice of the acquisition function $A$ and the updates of the local region in Section 4.1.

For the acquistion function $A$, we use Thompson sampling[1] as the acquisition function due to its compatibility with the discrete nature of our safety estimation and search space, and its innate ability for batch optimization by sampling the GP posterior.

 The local region $L_t$ is a trust region defined by a center (using current best feasible point) and a side length $l$, where we dynamically update the search space based on the sample history. Thank you for pointing out the confusion part and we would improve the presentation and add some illustration figures or psudo-codes to help readers better understanding our paper.

Reviewer comment (Experiments):
Considering the mapping from the original state space to the latent space is a key contribution, the author should explain more about why IRVAE is better in 4.1, and indicate where the related experimental results are.

Author response:
**Further analysis about IRVAE.**

We show the experiment of analyzing the IRVAE in Section 6.3. And we will improve our presentation in Section 4.1 to lead readers to the experiment part.

In Figure 4, we show that when using IRVAE, the GP estimation difference between original and latent space is significantly smaller than using standard VAE. We further compute point-wise distance of 10,000 random sampled points. The results shows high linear correlation of point-wise distance between original space and latent space ($R^2$ is 0.851 for musculoskeletal model control task and 0.973 for neural stimulation task), indicating good distance preserving ability of IRVAE.

Reviewer comment (Theory):
Considering the assumptions, it would be quite helpful if the authors could have a more general description of how we can apply the methods in the real world. Besides, why does the real clinical therapy optimization in Section 6.1 satisfy the assumptions?

Author response:
**General recipe of applying OLLSO in real-world applications.**

We believe OLLSO can be applied in various high-dimensional safety-critical problems. As mentioned above, the regularity assumptions are common-used in BO literature. In practice, we can fit the kernel hyperparameter using the observed data to make GP better model the functions. Many high-dimensional problems often have low-dimensional intrinsic structure, which satisfy the low effective dimension assumption. The distance preserving assumption depends on the embedding way, where using a IRVAE is able to approximately satisfy the condition.

In the real clinical therapy optimization, most dimensions (16 contact configurations) of the original input space are discrete, which is a sparse representation, and has the potential to reduce to a more compact representation. Our experimental result of IRVAE also shows that it can well preserve the distance and reconstruct the original input. Note that this exam of distance preserving is conducted before real evaluation, and we can start the real experiment after ensure the embedding satisfy the distance preserving requirement.

Overall, to apply OLLSO, the first step is to train an IRVAE using pre-collected or synthesized unlabelled data without actual evaluation. After verifying the embedding distance preserving capability, we can safely optimize the objective function following Algorithm 1. It is easy to be applied to other real-world problems.

Reviewer comment (Theory):
After Equation (1), I do not think $\alpha$ smaller than 0.5 will make the solution safe. Maybe I misunderstand some things. In Algorithm 1, there is an extra half bracket in the line 9.

Author response:
**Safety probabilistic threshold.**

Thank you for pointing out the typos. We will fix them in the next version.

In many real-world applications, we can exclude a large number of unsafe positions using prior knowledge. The remaining search region may have a certain probability to be safe even under random search. 
Therefore we set the probabilistic threshold smaller than 0.5, which is an optimistic estimation of the safety function in the problem formulation, and enables to use UCB to optimistically identify the safe region. We believe this formulation offers a more practical way to efficiently and safely optimize high-dimensional real-world problems.
Our experimental result also shows that OLLSO usually achieves safety probability larger than 80\% in the whole optimization procedure, and significant less constraint violation compared to other competitive baselines, which empirically demonstrates the safety guarantee when applying OLLSO in the real-world applications.

Reviewer comment (Novelty):
Even though the work is quite different from the safe exploration in RL [1], it would be great if the authors could have some discussion about their differences in terms of problem setting, safety mechanism, etc. The researchers in Safe RL would quite appreciate these discussions.

Author response:
**Discussion with safe exploration in RL**

Thank you for listing the related work about safe exploration in RL.
We will include the discussion of safe exploration in RL [2] in the next version.

Reviewer comment (Theory):
__Assumption 1:__ While this is a common assumption, _it is conventionally made on the original input space_. Assuming that the VAE latent space adheres to this assumption would entail that the projections from $\mathcal{Z}$ are sufficiently well-behaved. In vanilla BO, the modeling choices adhere to the assumption through the use of an appropriate kernel (Matern, RBF). However, it is not clear whether the decoding step is well-behaved (and due to its distortion of the space, I would be surprised if it was), so that _the assumption is adhered to_ in the proposed setup. I suggest the authors re-visit this assumption, since its correctness hinges on the continuity of the decoder. What makes them believe that the norm-bound $B$ is preserved through the decoding step? Since the theoretical analysis hinges on Assumption 1, which is decidedly _not_ common in latent space BO, the gap between the conventional assumption and the one made in latent space should be theoretically justified.

Author response:
**Regularity assumption over latent space.**

Our regularity assumption over the latent space enables to use GP to model function over the latent space. We agree that the norm-bound can not be preserved over the original space when the decoder is not well-behaved. Therefore, we want to use an isometric regularized autoencoder to achieve approximated distance preserving over the latent space. Our theoretical result(Theorem 1) is built upon both the regularity assumption(Assumption 1) and the distance preserving ability(Definition 1) of the latent embedding.

Reviewer comment (Theory):
__Assumption 3:__ The mapping $U$ is not currently properly defined, since it does not have an output. I believe you mean that $U$ is a _matrix_ or a map $R^D --> R^{d_e}$ (as in HeSBO). Further, wouldn't $g(\mathbf(x))$ imply that $g$ accepts D-dimensional input and $g(\mathbf(Ux))$ imply that $g$ accepts $d_e$-dimensional input?

Author response:
**Assumption about the effective dimension.**

We apologize for the not explicitly defining the low effective dimension of the original function. The correct formulation should be 

$g(\boldsymbol{x})=g(\boldsymbol{z}U^T)$

where $\boldsymbol{z}=U\boldsymbol{x}$, and $U^T$ is the inverse mapping from latent space to the original space.

Reviewer comment (Theory):
__Subsequent subspace assumption (After A3, henceforth AA3):__ _"Here we assume that the latent space mapping is an ε-subspace mapping "_. Please re-state this as a proper assumption for clarity. Moreover, is there a fundamental reason to believe this holds true, i.e. can properties of the IRVAE shed light on this? In HeSBO, the construction of the algorithm ensures this holds true. I don't see why the VAE would satisfy this unless it employed only linear layers (which I assume it does not).

Author response:
**$\epsilon$-subspace embedding assumption**

We agree that the $\epsilon$-subspace embedding assumption may not hold for standard VAE with non-linear layers. But we argue that IRVAE may approximately satisfied this assumption. In figure 4, we show that the GP estimation difference between original space and latent space of IRVAE is significantly smaller than latent space of standard VAE. 
We also compute point-wise distance of 10,000 random sampled points. The results shows high linear correlation of point-wise distance between original space and latent space ($R^2$ is 0.851 for musculoskeletal model control task and 0.973 for neural stimulation task), indicating good distance preserving ability of IRVAE.

Reviewer comment (Theory):
The intermediary conclusion, _"we can properly choose $\beta_t$ that satisfies the probability safety guarantee, under the noise-free setting"_ does not mesh well with __Assumption 2__ - observations are perturbed by noise.

Author response:
**Assumptions about noise observation.**

We propose OLLSO to optimize general safety-critical problems with noisy observation. In this paper, we theoretically derived the safety guarantee under noise-free setting, and we believe under bounded noise assumption, the derivation of safety guarantee under noise-perturbed observation is feasible.

We also empirically demonstrate that OLLSO works well under noisy setting in the clinical experiment, where the observation is perturbed by noise from measuring EMG signal of lower limb muscles.

Reviewer comment (Theory):
Due to the three (A1, A3, AA3) seemingly unverifiable and seemingly incorrect (A1, AA3) assumptions that are made, I do not attribute any value to the theory that is being presented. I encourage the authors to shed light on why these assumptions are valid.

Author response:
**The novelty of the method.**

As mentioned above, we argue that the assumptions we use in this paper are not convenient assumptions. The regularity assumptions are the prerequisite of all latent space BO methods, and the $\epsilon$-subspace assumption can be satisfied using some linear projection or approximately satisfied by some distance-preserved autoencoder. We believe our result of safety guarantee is novel.

Our experiment also empirically verifies our theoretical result, where OLLSO consistently outperforms existing constrained BO methods or safe BO methods in terms of both optimization performance and safety guarantee. Our clinical experiment also demonstrates the success of OLLSO in optimizing real-world high-dimensional safety-critical problems. We believe that our practical contribution is novel.

Overall, we believe that our proposed method is novel in both algorithmic and practical aspects, and is both practical and theoretically verifiable to safely and efficiently optimize real-world safety-critical problems.

[1] Ryan-Rhys Griffiths and José Miguel Hernández-Lobato. Constrained Bayesian optimization for automatic chemical design using variational autoencoders. Chemical Science, 2020.

[2] M. A. Gelbart, J. Snoek and R. P. Adams, Bayesian optimization with unknown constraints. Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, 2014.

[3] Tripp, Austin, Erik Daxberger, and José Miguel Hernández-Lobato. "Sample-efficient optimization in the latent space of deep generative models via weighted retraining." Advances in Neural Information Processing Systems 33 (2020): 11259-11272.

[4] Grosnit, Antoine, et al. "High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning." arXiv preprint arXiv:2106.03609 (2021).

Reviewer comment (Novelty):
__Novelty of method:__ OLLSO combines existing VAE architectures with trust regions, Thompson sampling and constraints. VAEs for BO have been extensively explored, and the other components are exactly SCBO (Eriksson and Poloczek 2021). While the combination is interesting, no novel building blocks for BO are proposed.

Author response:
**The novelty of the method.**

In this paper, we propose a practical method to achieve safe and efficient optimization of high-dimensional functions. To our best knowledge, no existing work focuses on guaranteeing safety during the high-dimensional optimization.

We present the novelty of OLLSO in both algorithmic and practical aspects.

In terms of latent space optimization, previous latent BO works neglect the geometry-preserving during unsupervised representation learning, which is crucial to extend safety guarantee from latent space to the original space. Many of them achieve latent space shaping using collected labeled data [1, 2]. We are the first to use distance-preserved VAE to address safety issue in latent space optimization, and derive the theoretical probabilistic safety guarantee of our proposed method.

The local region adaptation of OLLSO is also different from existing 
trust-region based BO methods[3, 4] which neglect the safety during the optimization procedure. They would also discard all previous sample point when restarting the trust region. OLLSO updates the trust region in a more safety-sensitive way, and keeps all previous samples when restarting the trust region, ensuring a safer optimization procedure with theoretical guarantee.

In terms of the safe optimization part, previous GP-based safe optimization method are limited to low-dimensional problems (typically below 10, [5, 6]). OLLSO addresses this issue by introducing local search mechanism upon optimistic safety region identification, and is able to handle problem with hundreds of dimensions. 

Our experimental result also shows that OLLSO is not a direct application to comprise exsiting algorithms. Figure 2 and Figure 3 show that simply combining IRVAE with existing safe BO (CONFIG (L)) or trust region-based constrained BO (SCBO (L)) perform worse than OLLSO in terms of both optimization performance and safety guarantee. Our simulation results shows that OLLSO significantly outperforms existing safe BO methods, constrained BO algorithms, and direct combinations with dimension reduction models.

We want to emphasize that OLLSO is a PRACTICAL method that can safely optimize high-dimensional real-world problems. In addition to state-of-the-art simulation performance, our real clinical experiments show that OLLSO successfully optimizes neural stimulation for the control of human muscles. While the majority of our cited papers conduct their experiments only in simulation or synthetic problems, OLLSO is both practical and theoretically verifiable to safely and efficiently optimize real-world safety-critical problems.

Reviewer comment (Writing):
__VAE-related background:__ BO and GPs are well-covered in the background, but the VAE-related design choices are not. As it is a vital part of the algorithm, background on the IRVAE is integral to the paper.

Author response:
**VAE-related background**

Thank you for pointing out the missing discussion of this topic. We will include it in the next version of the paper.

Reviewer comment (Presentation):
__Parsing of results:__ Plots are difficult to process due to the small fontsize, and text which overlaps with the rest of the plot (Fig. 4). Consider making it substantially larger, possibly at the expense of moving some of it to the Appendix.

Author response:
**Parsing of results and typos**

Thank you for finding typos in our paper. We will correct them in the next version.

Reviewer comment (Theory):
$u_t(z)$: If the $\max C$ is over $z$ (which it should be) then $u_t$ is not a function of $z$.

Author response:
**The upper confidence bound**

We define $u_t(\boldsymbol{z})\coloneqq\max C_t(\boldsymbol{z})$, where $C_t(\boldsymbol{z})\coloneqq [\mu_t(\boldsymbol{z})\pm\beta\sigma_t(\boldsymbol{z})]$ is the confidence interval at the position $\boldsymbol{z}$. Therefore, the max operation is performed over the interval instead of $\boldsymbol{z}$.

Reviewer comment (Theory):
_"Our approach resets l to its initial length, ensuring a different, safer trajectory sampling than the initial instance."_ Why does this action _ensure_ a safer trajectory?

Author response:
1. Explanation of safer trajectory after reset.

When resetting the trust region, we preserve all sample points compared to SCBO, which discards all history data. In this way, we have more knowledge about the safety function compared to last reset point, and the future sample procedure will be safer.

Reviewer comment (Writing):
Algorithm 1, line 2: What is the _trajectory_ $\zeta_0$?

Author response:
2. The sample trajectory

The sample trajectory $\zeta_{t}$ is defined as a set of tuples {$(\boldsymbol{x}_t,y_t^f,y_t^g)$}. The update of the trust region is based on the selection result of last round, which is a process depending on the historical selection trajectory.

Reviewer comment (Experiments):
For the trust region, how is $\ell$ set?

Author response:
3. The setting of trust region length $l$

We directly use the setting of SCBO BoTorch implementation(https://botorch.org/tutorials/scalable_constrained_bo). In concrete, we set the initial $l$ as 0.8, $l_{min}$ is $0.5^7$, and $l_{max}$ is 1.6. We normalize all inputs in the range of $[0,1]$, therefore we can use same setting across different tasks.

Reviewer comment (Experiments):
Why can unlabeled data be synthetized (in large quantities)?

Author response:
**Unlabeled dataset synthesizing**

Taking neural stimulation task as an example, we synthesize the unlabelled stimulation parameter by randomly setting the contact configuration and the intensity under clinical prior (e.g. limiting the contact number or intensity range). The procedure is capable of generating large quantity of data without actual evaluation. There are many other real-world applications where we can synthesize data in the same way, such as protein/molecular sequences.

Reviewer comment (Experiments):
Though the discussion involves random-projection-based dimension reduction methods, it lacks direct comparison with the proposed IRVAE, especially when the assumption is shared. Specifically, HESBO [1] offers a comprehensive discussion of the feasibility of $\epsilon$-subspace embedding and its impact on downstream uncertainty quantification when applying popular kernels. The recent follow-up work [2] further advances this direction by reducing the risk of losing global optimum in the embedding. A direct comparison could better motivate the proposed IRVAE, which lacks a similar theoretical guarantee and requires additional assumption 3 to proceed with the analysis.

Author response:
**Comparison with random-embedding BO**

Thanks for your suggestions on adding random-embedding-based methods for comparison. We follow your suggestion and run HesBO and BAxUS on a musculoskeletal model control task and a neural stimulation task with no safety constraints in simulation. Due to the algorithmic mechanism of HesBO and BAxUS, we cannot directly use the same initial point as OLLSO. Therefore we randomly sample initial points from their corresponding latent space. In HesBO, we set the same latent dimension number as in OLLSO. The following tables are the best objective function values found by algorithms (shown as mean $\pm$ 1 std).

| Algorithm | Muscle                         | SCS-IL                     | SCS-RF                     | SCS-TA                     |
|-----------|--------------------------------|----------------------------|----------------------------|----------------------------|
| OLLSO     | $\boldsymbol{284.08\pm305.95}$ | $\boldsymbol{0.39\pm0.03}$ | $\boldsymbol{0.37\pm0.00}$ | $\boldsymbol{0.38\pm0.02}$ |   |
| HesBO     | $-747.8\pm124.53$              | $0.34\pm0.03$              | $0.30\pm0.04$              | $0.36\pm0.05$              |
| BAxUS     | $-619.36\pm302.83$             | $0.36\pm0.04$              | $0.35\pm0.02$              | $0.34\pm0.05$              |

| Algorithm | SCS-BF                     | SCS-ST                     | SCS-GA                     |
|-----------|----------------------------|----------------------------|----------------------------|
| OLLSO     | $\boldsymbol{0.28\pm0.03}$ | $\boldsymbol{0.26\pm0.02}$ | $\boldsymbol{0.22\pm0.01}$ |
| HesBO     | $0.25\pm0.01$              | $0.23\pm0.0$               | $0.19\pm0.02$              |
| BAxUS     | $0.27\pm0.02$              | $0.24\pm0.03$              | $0.19\pm0.05$              |

We observe OLLSO still outperforms HesBO and BAxUS across all tasks, even when optimizing under safety constraint. We think using IRVAE enables utilizing the pre-collected unlabelled data to learn a better representation than random projection.

Reviewer comment (Novelty):
Given existing work in distance preserving dimension reduction, local optimization for efficient global optimization, and safe exploration for optimization with Gaussian processes, the general novelty of the proposed method is very limited.

Author response:
**The novelty of our work.**

In this paper, we propose a practical method to achieve safe and efficient optimization of high-dimensional functions. To our best knowledge, no existing work focuses on guaranteeing safety during the high-dimensional optimization.

We present the novelty of OLLSO in both algorithmic and practical aspects. 

In terms of latent space optimization, previous latent BO works neglect the geometry-preserving during unsupervised representation learning, which is crucial to extend safety guarantee from the latent space to the original space. Many of them achieve latent space shaping using collected labeled data [1, 2]. We are the first to use distance-preserved VAE to address safety issue in latent space optimization, and derive the theoretical probabilistic safety guarantee of our proposed method.

The local region adaptation of OLLSO is also different from existing 
trust-region based BO methods [3, 4] which neglect the safety during the optimization procedure. They would also discard all previous sample points when restarting the trust region. OLLSO updates the trust region in a more safety-sensitive way, and keeps all previous samples when restarting the trust region, ensuring a safer optimization procedure with theoretical guarantee.

In terms of safe optimization, previous GP-based safe optimization method are limited to low-dimensional problems (typically below 10, [5, 6]). OLLSO addresses this issue by introducing local search mechanism upon optimistic safety region identification, and is able to handle problem with hundreds of dimensions. 

Our experimental result also shows that OLLSO is not a direct application to comprise exsiting algorithms. Figure 2 and Figure 3 show that simply combining IRVAE with existing safe BO (CONFIG (L)) or trust region-based constrained BO (SCBO (L)) perform worse than OLLSO in terms of both optimization performance and safety guarantee. Our simulation results shows that OLLSO significantly outperforms existing safe BO methods, constrained BO algorithms, and direct combinations with dimension reduction models.

We want to emphasize that OLLSO is a PRACTICAL method that can safely optimize high-dimensional real-world problems. In addition to state-of-the-art simulation performance, our real clinical experiments show that OLLSO successfully optimizes neural stimulation for the control of human muscles. While the majority of our cited papers conduct their experiments only in simulation or synthetic problems, OLLSO is both practical and theoretically verifiable to safely and efficiently optimize real-world safety-critical problems.

[1] Tripp, Austin, Erik Daxberger, and José Miguel Hernández-Lobato. "Sample-efficient optimization in the latent space of deep generative models via weighted retraining." Advances in Neural Information Processing Systems 33 (2020): 11259-11272.

[2] Grosnit, Antoine, et al. "High-dimensional Bayesian optimisation with variational autoencoders and deep metric learning." arXiv preprint arXiv:2106.03609 (2021).

[3] Eriksson, David, et al. "Scalable global optimization via local Bayesian optimization." Advances in neural information processing systems 32 (2019).

[4] Eriksson, David, and Matthias Poloczek. "Scalable constrained Bayesian optimization." International Conference on Artificial Intelligence and Statistics. PMLR, 2021.

[5] Sui, Yanan, et al. "Safe exploration for optimization with Gaussian processes." International conference on machine learning. PMLR, 2015.

[6] Xu, Wenjie, et al. "Constrained efficient global optimization of expensive black-box functions." International Conference on Machine Learning. PMLR, 2023.

Reviewer comment (Presentation):
Could the author further clarify the results shown in Figure 3? What are the legends in (b) stand for? Does the bar plots in (c) show the cumulative results after 500 evaluation corresponding to the results in (b)?

Author response:
**Experiment result clarification**

1. Figure 3 (b) shows the clinical result in neural stimulation therapy optimization, where IL (iliopsoas), RF (rectus femoris), TA (tibialis anterior), BF (biceps femoris) are different groups of target muscles on the lower limb, which are the main muscle groups that control motor funuction. The "L" (Left) or "R" (Right) after the underline are the sides of human lower limbs. 

2. Figure 3 (c) shows the simulation result using a computational spinal cord model, where the experiment result is detailed in Section 6.3.1. Bar plot in Figure 3 (c) shows the best feasible objective function value (Objective, top), safe decision ratio of all samples (Safe \%, middle), and cumulative safety violation (Violation, bottom) after 800 samples.

Thank you for pointing out the confusion part, and we will improve our experiment result presentation in the next version.

Reviewer comment (Theory):
The purpose and implications of the current theoretical analysis are unclear. Theorem 1 is said to translate a probabilistic constraint on the latent space to another on the original space, but it seems to instead suggest a translation on the UCB of the safety function prediction.

Author response:
**Link the theoretical result with the optimization objective.**

In Theorem 1, we prove that with proper choice of the confidence bound scalar $\beta$, the selected safety value is lager than the estimated upper confidence bound with a probability larger than $\alpha$ (that is, $Pr(g(\boldsymbol{x}_t) \geq u(\boldsymbol{x}_t))\geq\alpha$). During optimization, we only consider points whose UCB is larger than the safety threshold $h$ as candidates (Line 7 in Algorithm 1). In this way we can guarantee that the selected safety value is lager than the safety threshold $h$ with a probability larger than $\alpha$ (that is, $Pr(g(\boldsymbol{x}_t) \geq h)\geq Pr(g(\boldsymbol{x}_t) \geq u(\boldsymbol{x}_t)) \geq \alpha)$.

Reviewer comment (Writing):
The algorithmic exposition is unclear at several points. For example, in Algorithm 1 (line 6), it is not clear how L_t is defined, nor how it should be updated in line 6.

Author response:
**Algorithm 1 explained in more mathematical details.**

* (Line 1) we train an IRVAE using the unlabelled dataset $D_u^{\mathcal{X}}$ by adding an isometric regularization loss term over the original loss function (Line 1). Our training objective is 

    $\min_{\theta} \mathcal{L}(\theta) + \mathcal{F}(\theta)$,

where $\theta$ is the parameters of the IRVAE, $\mathcal{L}(\theta)$ is the original loss function (typically using evidence lower bound in VAE), and $\mathcal{F}(\theta)$ is the additional regularization term proposed by [1] to shape the learned representation towards isometric.

* (Line 2-3) We initialize the sample trajectory as an empty set, and start optimization loop. The sample trajectory $\zeta_{t}$ is defined as a set of tuples {$(\boldsymbol{x}_t, y_t^f, y_t^g)$}.

* (Line 4-5) At each optimization round $t$, we use the trained IRVAE to map original sampled data to the latent space, and update the objective and safety GP model using Eq.2. 

* (Line 6) We update the local region $L_t$ based on the sample trajectory $\zeta_{t-1}$. $L_t$ is a trust region defined by a center (using current best feasible point) and a length $l$.  We denote successful/failure of the optimization round using the last sample result: A sampling round is considered "successful" if it finds a better reward while maintaining comprehensive safety. Conversely, it is labelled a "failure" if any unsafe points are found or if there is no discernible improvement. The side length is adjusted—increased ($l_t = 2l_{t-1}$) for successes and decreased ($l_t = 0.5l_{t-1}$) for failures—upon reaching a preset threshold.

* (Line 7-8) We use UCB of the safety GP to identify the safe region, and maximize the acquisition function (using TS here) to get next latent sample. 

* (Line 9-12) We project the latent sample back to original space, evaluate the function values and update the sampled dataset and sample trajectory. 

We will improve our presentation to help readers better understand our proposed method.