Paper: PAE: Reinforcement Learning from External Knowledge for Efficient Exploration (ICLR-2024)

Reviewer comment (Experiments):
The PAE model comprises three primary components, each with distinct hyper-parameters. Managing these varied parameters can be challenging in real applications.

Author response:
We fully understand your concerns. As mentioned in Appendix A.4.1, our implementation of PAE and the reproduction of the baseline algorithm both utilize TorchBeast. Consequently, most hyperparameters align with those in TorchBeast's version, minimizing the need for extensive tuning. For new modules or structures unique to PAE, we conducted a grid search to determine their hyperparameters. The scope of this search and the final hyperparameters are detailed in Appendix A.3.4. Additionally, we have uploaded our source code in the appendix to facilitate easy reproduction of our work by other researchers.

Reviewer comment (Evaluation):
Since it depends on special environments and predefined knowledge template, PAE is not generalizable to a broader range of tasks.

Author response:
To address your concerns, we conducted additional experiments on MiniHack\[1\] to demonstrate that **PAE is scalable and does not rely on predefined knowledge templates**. MiniHack has a more complex state and action space and also possesses direct access to real natural language feedback ([Screenshot of Minihack Language](https://ibb.co/mDzz6Rm)) that human players receive during gameplay. We incorporate this systematic feedback into the PAE as external knowledge, making it more natural and realistic, rather than relying on specific templates. For additional information on the MiniHack task, please refer to Section 5 and Appendix A.2 of the revised version.

We conducted comparisons between PAE and the main baselines across two task types, **totaling five environments within the MiniHack**. For your convenience, we have included a list of the results below and training curves ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP)). For detailed information on the new MiniHack tasks and additional experimental results, please refer to Section 5 and Appendix A.2 in our revised version. 

Table 1:  Comparison of PAE and baseline methods in MiniHack tasks. (mean ± std, convergence steps)

|           | LavaCross-Ring        | LavaCross-Potion      | LavaCross-Full        | Mutiroom-N4-Monster   | River-Monster         |
|-----------|:-----------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:-----------------------:|
| Ours      | 1.00±0.001, 22M   | 0.99±0.002, 35M   | 1.00±0.001, 24M   | 0.72±0.020, >40M  | 0.13±0.018, >20M  |
| L-AMIGo   | 0.57±0.223, >40M  | 0.58±0.214, >40M  | 0.45±0.111, >40M  | 0.31±0.035, >40M  | 0.09±0.016, >20M  |
| AMIGo     | 0.46±0.116, >40M  | 0.54±0.048, >40M  | 0.44±0.101, >40M  | 0.29±0.026, >40M  | 0.08±0.009, >20M  |
| ICM       | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.18±0.199, >40M  | 0.12±0.028, >20M  |
| IMPALA    | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >20M  |
| RND       | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >20M  |

Each entry in Table 1 consists of two numbers: the first denotes the average extrinsic reward achieved, and the second represents the minimum number of stabilizing steps required to achieve a reward plateau, where further significant increases are no longer observed. **The results demonstrate that our PAE approach exhibits good scalability and performs well without relying on predefined knowledge, i.e., without specific templates.**

Reviewer comment (Evaluation):
What is the difference in wall-clock running time between PAE and the other baseline models?

Author response:
Thank you very much for your suggestion. Wall-clock running time is indeed crucial for evaluating algorithm performance. 
To ensure a fair comparison and exclude interference, all algorithms must run independently on the same configured machines. Due to time constraints, we only report the wall-clock running time of different algorithms on KeyCorrS5R3 in Table 1.

Table 2: Wall-clock running time for PAE and other baseline algorithms on KeyCorrS5R3

|  Model  |  PAE (Ours)  |  L-AMIGo  |  AMIGo  |  ICM  |  IMPALA  |  RND  |
| --- | --- | --- | --- | --- | --- | --- |
|  Time (hours)  |  18.9   |  17.3   |  25.7   |  20.0   |  11.8   |  20.7   |

As demonstrated in Table 2, our PAE method requires comparable training time to the baselines. This similarity is due to both our PAE and baseline algorithms being built on the TorchBeast implementations, with our method not introducing significant additional computation. Therefore, it does not significantly increase wall-clock running time.

Reviewer comment (Experiments):
Would it be possible to extract latent knowledge from a collected dataset in a pre-training stage and replace the template-based knowledge in PAE?

Author response:
Yes, It would be. In fact, it aligns with our current approach, as mentioned in Section 4.1, where we utilize a pre-trained language model (i.e., BERT-base) to encode and generate knowledge embeddings. We use template-based knowledge solely because the BabyAI environment only provides this type of knowledge. In our further experiments with the new MiniHack environment, we incorporate real natural language feedback from human players during gameplay as external knowledge, eliminating the need for specific templates. The supplemental experiments show that our PAE approach remains superior. Detailed experimental results and non-template-specific knowledge from MiniHack are presented in Section 5 and Appendix A.2 of the revised version.

Reviewer comment (Experiments):
As authors pointed, the current knowledge relies on the specific template, which lacks naturalness and does not fully use the huge ability of the LLM. It is an obvious weakness for this paper.

Author response:
We fully understand your concerns. We mainly focus on how external knowledge guides agent exploration and therefore directly adopt the pre-defined knowledge set provided by BabyAI without additional processing.

To address your concerns, we have extended our PAE approach to the more challenging MiniHack task \[2\]. In addition to having a more complex state space and action space, MiniHack provides direct access to real natural language feedback ([Screenshot of Minihack Language](https://ibb.co/mDzz6Rm)) received by human players during gameplay. We integrate this feedback into PAE as external knowledge, **enhancing its naturalism and realism without the need for specific templates**. For more information on MiniHack tasks, see Section 5 and Appendix A.1 in the revised version.

We conducted comparisons between PAE and the main baselines across two task types, **totaling five environments within the MiniHack**. For your convenience, we have included a list of the results below and training curves ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP)). For detailed information on the new MiniHack tasks and additional experimental results, please refer to Section 5 and Appendix A.2 in our revised version. 

Table 1:  Comparison of PAE and baseline methods in MiniHack tasks. (mean ± std, convergence steps)

|           | LavaCross-Ring        | LavaCross-Potion      | LavaCross-Full        | Mutiroom-N4-Monster   | River-Monster         |
|-----------|:-----------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:-----------------------:|
| **Ours**      | **1.00±0.001, 22M**   | **0.99±0.002, 35M**   | **1.00±0.001, 24M**   | **0.72±0.020**, >40M  | **0.13±0.018**, >20M  |
| L-AMIGo   | 0.57±0.223, >40M  | 0.58±0.214, >40M  | 0.45±0.111, >40M  | 0.31±0.035, >40M  | 0.09±0.016, >20M  |
| AMIGo     | 0.46±0.116, >40M  | 0.54±0.048, >40M  | 0.44±0.101, >40M  | 0.29±0.026, >40M  | 0.08±0.009, >20M  |
| ICM       | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.18±0.199, >40M  | 0.12±0.028, >20M  |
| IMPALA    | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >20M  |
| RND       | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >40M  | 0.00±0.000, >20M  |

Each entry in Table 1 consists of two numbers: the first denotes the average extrinsic reward achieved, and the second represents the minimum number of stabilizing steps required to achieve that reward. The results indicate that our PAE method performs superiorly without relying on predefined knowledge (i.e., no specific templates). The results of our complementary experiments have addressed the limitation regarding specific templates, and we have updated the discussion in the revised version.

Reviewer comment (Experiments):
The current knowledge needs human labeling following the specific template, which is of labor consumption.

Author response:
We apologize for the confusion caused by our unclear presentation. Currently, knowledge in BabyAI is automatically provided by the environment and doesn't require additional human labeling. To address your concerns further, we've added the MiniHack task, which provides real natural language feedback ([Screenshot of Minihack Language](https://ibb.co/mDzz6Rm)) received by the agent during gameplay, and **no specific template requires human labeling**. Extended experiments in MiniHack demonstrate that our PAE can handle non-template-specific knowledge in natural language form. You can find the list of knowledge in MiniHack and the experiment results in Appendix A.2 of the revised version.

Reviewer comment (Experiments):
The authors use the cross-attention as the alignment, how about the current alignment work like CLIP and ImageBind? Do the authors consider to use this alignment?

Author response:
This is an interesting and worth exploring question. If we go a little deeper into the details, we can **unify** the mechanism of alignment in PAE, CLIP, and ImageBind. In PAE, we employ Scaled-CrossAttention for aligning knowledge $\textbf{k}$ and state $\textbf{s}$ , generating the Attention Map between the Query $Q=W\_Q\cdot \textbf{k}$ and the Key $K=W\_K\cdot \textbf{s}$ . CLIP and ImageBind achieve cross-modal ( $\textbf{m}_1$ and  $\textbf{m}_2$) alignment by utilizing a similar Key-Query manner, generating an Relation Map between the Query $Q=W\_Q\cdot \textbf{m}_1$ and the Key $K=W\_K\cdot \textbf{m}_2$ .

 The primary distinction among these three methods lies in their use of different normalization techniques. 

 $\text{PAE}: \quad  \quad \quad \text{Attention} = \text{SoftMax}(Q\cdot K^T/ \sqrt{d\_k})$

 $\text{CLIP}: \quad \quad \quad \text{Relation} = \text{SoftMax}(\text{norm}(Q)\cdot \text{norm}(K^T)) \quad  $

 $\text{ImageBind}: \quad \text{Relation} = \text{SoftMax}(Q\cdot K^T/\tau)  $

Specifically, PAE normalizes using the dimension of the Key vector, while CLIP normalizes both Key and Query, and ImageBind employs temperature coefficients $\tau$ for normalization. 

CLIP and ImageBind may offer a significant advantage as they are pre-trained on large natural image datasets. However, there is a domain gap between natural images and unrealistic virtual game graphics. More importantly, our environments (BabyAI and MiniHack) and various baselines all use vectorized inputs, so CLIP and ImageBind are not suitable for direct porting to our task, which is why we chose the learnable Scaled-CrossAttention alignment mechanism. Thanks to your insights, we have included a discussion on these two approaches in Appendix A.1 of revised version.

Reviewer comment (Experiments):
In Figure 4, I think there is unfair to compare with most of these baselines, since they do not have or use the language guidance. The L-AMIGo is the only one considering language. Are there any other similar work that the authors can compare? The current comparison is not very strong to highlight your work.

Author response:
Thank you very much for your valuable suggestion. We have followed your suggestion and introduced another recent Language-Instructed Algorithm called L-NovelD \[3\] for comparison with our PAE method. L-NovelD combines natural language with an intrinsic motivation approach to reward states described in natural language that transitions from low to high novelty. Mu et al. \[3\]  demonstrated that L-NovelD outperforms several popular intrinsic motivation methods, including NovelD, RND, and others, in the BabyAI tasks. Detailed information about the L-NovelD method can be found in Section 5 and Appendix A.4.2 of the revised version. 

We conducted a comparative analysis of the PAE and L-NovelD methods across the six BabyAI environments. The table below and training curves ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP)) present results for the added L-NovelD and PAE comparisons. For a comprehensive view of experimental results and training curves, please refer to Section 5 of the revised version. 

Table 2: Comparison of PAE and baseline methods (mean ± std, convergence steps)

|  Model  |  KEYCORRS3R3  |  KEYCORR43R3  |  KEYCORRS5R3  |  OBSTRMAZE1D1  |  OBSTRMAZE2D1HB  |  OBSTRMAZE1Q  |
| --- | --- | --- | --- | --- | --- | --- |
|  **Ours**  |  **0.89**$\pm$**0.002**, **6M**  |  **0.92**$\pm$**0.005**, **30M**  |  **0.94**$\pm$**0.001**, **90M**  |  **0.93**$\pm$**0.004**, **6M**  |  **0.87**$\pm$**0.018**, **150M**  |  **0.89**$\pm$**0.006**, **150M**  |
|  L-AMIGo  |  0.86 $\pm$ 0.040, 12M  |  0.90 $\pm$ 0.011, 60M  |  0.93 $\pm$ 0.016, 160M  |  0.90 $\pm$ 0.030, 20M  |  0.23  $\pm$0.403, \>200M  |  0.47 $\pm$ 0.029, \>300M  |
|  L-NovelD  |  0.63  $\pm$0.097, \>20M  |  0.15 $\pm$ 0.195, \>60M  |  0.55 $\pm$ 0.484, \>200M  |  0.79 $\pm$ 0.255, \>20M  |  0.57 $\pm$ 0.164, \>200M  |  0.60 $\pm$ 0.107, \>200M  |
|  AMIGo  |  0.43 $\pm$ 0.246, \>20M  |  0.22 $\pm$ 0.343, \>60M  |  0.43  $\pm$0.599, \>200M  |  0.45  $\pm$0.366, \>20M  |  0.10 $\pm$ 0.175, \>200M  |  0.18 $\pm$ 0.023, \>300M  |
|  ICM (Curiousity)  |  0.04 $\pm$ 0.052, \>20M  |  0.30 $\pm$ 0.511, \>60M  |  0.00  $\pm$0.000, \>200M  |  0.31  $\pm$0.542, 15M  |  0.00  $\pm$0.000, \>200M  |  0.00 $\pm$ 0.000, \>300M  |
|  IMPALA  |  0.00 $\pm$ 0.000, \>20M  |  0.00 $\pm$0.000, \>60M  |  0.00 $\pm$ 0.000, \>200M  |  0.00 $\pm$ 0.006, \>20M  |  0.00 $\pm$ 0.000, \>200M  |  0.00 $\pm$ 0.000, \>300M  |
|  RND  |  0.00 $\pm$ 0.000, \>20M  |  0.00 $\pm$ 0.000, \>60M  |  0.00 $\pm$ 0.000, \>200M  |  0.00 $\pm$ 0.000, \>20M  |  0.00 $\pm$ 0.000, \>200M  |  0.00 $\pm$ 0.000, \>300M  |

\*Due to very limited time, we have so far completed the adaptation and training of L-NovelD on 6 environments in BabyAI, and the training in the 5 additional MiniHack environments is still running. We will update our full experiments in the camera ready version.

Each entry in Table 1 consists of two numbers: the first denotes the average extrinsic reward achieved, and the second represents the minimum number of stabilizing steps required to achieve a reward plateau, where further significant increases are no longer observed. The results demonstrate that our PAE method still outperforms the L-NovelD method, which also utilizes natural language as an aid. 

**Ref :**

\[1\] Alec Radford, et al. "Learning transferable visual models from natural language supervision." ICML2021.

\[2\] Rohit Girdhar, et al. "Imagebind: One embedding space to bind them all." CVPR2023. 

\[3\] Jesse Mu, et al. "Improving Intrinsic Exploration with Language Abstractions." NeurIPS 2022.

Reviewer comment (Experiments):
BabyAI action space is quite limited (6 actions) and simplistic, so are the tasks since they are 2D grid world tasks.

Author response:
To address your concerns, we have extended our PAE approach to the **more challenging** MiniHack tasks \[1\]. MiniHack consists of procedurally generated tasks within a roguelike game, offering **a richer observation space** and up to **75 dimensions of structured and context-sensitive action space**, presenting huge challenges. The high complexity of this state-action space makes MiniHack **one of the most challenging sparse reward environments** to explore \[2\]. We conducted comparisons between PAE and the main baselines across two task types,**totaling five** **environments** (**LavaCross-Ring, LavaCross-Potion, LavaCross-Full, MultiRoom-N4-Monster, River-Monster**) within the MiniHack. For ease of comparison, we have included a list of the results below and training curves ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP)). For detailed information on the new MiniHack tasks and additional experimental results, please refer to Section 5 and Appendix A.2 in our revised version. 

We have also provided ample experimental results to demonstrate the effectiveness and scalability of our work. Particularly, we have shown that our approach can be readily extended to more challenging environments.

Table 1: Comparison of PAE and baseline methods in MiniHack tasks. (mean ± std, convergence steps)

| | LavaCross-Ring | LavaCross-Potion | LavaCross-Full | Mutiroom-N4-Monster | River-Monster |
|-----------|:-----------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:-----------------------:|
| **Ours** | **1.00±0.001, 22M** | **0.99±0.002, 35M** | **1.00±0.001, 24M** | **0.72±0.020**, >40M | **0.13±0.018**, >20M |
| L-AMIGo | 0.57±0.223, >40M | 0.58±0.214, >40M | 0.45±0.111, >40M | 0.31±0.035, >40M | 0.09±0.016, >20M |
| AMIGo | 0.46±0.116, >40M | 0.54±0.048, >40M | 0.44±0.101, >40M | 0.29±0.026, >40M | 0.08±0.009, >20M |
| ICM | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.18±0.199, >40M | 0.12±0.028, >20M |
| IMPALA | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >20M |
| RND | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >40M | 0.00±0.000, >20M |

Each entry in Table 1 consists of two numbers: the first denotes the average extrinsic reward achieved, and the second represents the minimum number of stabilizing steps required to achieve a reward plateau, where further significant increases are no longer observed. The results indicate that our PAE method performs superiorly and is easy to be adapted to more challenging environments.

Reviewer comment (Experiments):
As pointed out by authors, current experiments have a knowledge oracle and a very specific template for knowledge pieces. Also, planners can observe the whole world, which may be unrealistic for many real-world set up.

Author response:
Thank you very much for your comments. The second reason we conducted experiments on the MiniHack task is that, MiniHack provides **natural language knowledge** ([Screenshot of Minihack Language](https://ibb.co/mDzz6Rm)) which is real-world feedback received by human players or agents during gameplay, rather than specific knowledge template in BabyAI. Extended experiments in MiniHack ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP)) demonstrate our PAE's ability to handle non-template-specific knowledge in natural language form and. its effectiveness, versatility, and scalability. A list of the knowledge from MiniHack and the experiment results can be found in Section 5 and Table 4 in the revised version. 

We mention these limitations in the manuscript because our research originally focused on how external knowledge guides agent exploration. So, it **directly followed the templates provided by BabyAI to express knowledge**. The results of our latest complementary experiments have addressed the limitation regarding specific templates, and we have updated more discussions in the revised version accordingly. 

Besides, we are very grateful for your suggestions to our work, which have significantly contributed to our exploration of the boundaries of the PAE's capabilities. Thank you very much.

This is actually the default setting of the baseline methods in the BabyAI environment, and we strictly followed this setting for a fair comparison. To address your concern, in our newly introduced MiniHack task, both **the Planner and Actor utilize identical partial observation inputs.** This uniform setting is also maintained across all other baselines. Experimental results confirm that our PAE method can still consistently outperform other baselines.

Reviewer comment (Experiments):
Current experiments require well-defined sub-tasks as well as detection of sub-task completion. Baby AI tasks are about navigating a grid in a particular way and due to task and world structure, it is easy to detect sub-task completion.

Author response:
Thank you very much for your comments. You are correct in understanding that the current experiment involves explicit sub-tasks and the detection of sub-task completion.  In fact, most of the subtasks provided in the BabyAI environment do not contribute to the completion of the final task, which, on average, only requires 6 to 12 messages (1-2% of all 652 messages) to complete \[3\]. The key to earning the final reward lies in **identifying useful subtasks and establishing the sequential relationships between them**. 

In Figure 5 of our paper, we visually demonstrate how PAE progressively learns this logical relationship, and Figure 10 illustrates the process that agents across various tasks must follow to achieve the final reward. Despite intrinsic motivation methods like AMIGo, L-AMIGo, etc., **also receiving cues about sub-tasks and sub-task completion detection, our PAE methods outperform them**, as indicated in Figure 4. 

Additionally, we claim that sub-tasks, like certain actions triggering specific text messages, are more and more common among games, namely "in-game guidance messages". These text messages, which already embedded in the status information, may not be fully utilized by previous baseline methods, limiting their performance. While in our work PAE, the Planner's role is to automatically align and organize these sub-tasks into its curriculum, aiding the Actor in better assimilating external knowledge and exploring the environment.

Last but not least, our new MiniHack environment includes **Skill Acquisition Tasks, distinct from the navigation tasks in BabyAI** with less structured text knowledge, yet our PAE excels in grasping the sequential relationship between subtasks to complete the final task, surpassing other baselines. backing its effectiveness.

Reviewer comment (Theory):
This method would need sub-task and knowledge alignment so as to detect if a particular piece of knowledge lead to completion of a particular sub-task. I don’t see how it can work any other way. This is a very hard thing to obtain.

Author response:
We fully understand your concerns and apologize for any confusion arising from the lack of clear explanations in our paper. External knowledge indeed contributes to subtask completion detection, but this is not the full and crucial role played by external knowledge. Despite intrinsic motivation methods like AMIGo, L-AMIGo, etc., also receiving cues about sub-tasks and sub-task completion detection, our PAE methods outperform them, as indicated in Figure 4. The core innovation of our paper is how to utilize external knowledge to enhance the agent's exploration capabilities. We introduce a generalized framework, PAE: the **Planner** utilizes external knowledge for creating a progressive curriculum through feedback and guidance; the **Actor** aligns its current capabilities with the environment state with external knowledge's assistance, and the **Evaluator** employs external knowledge to detect subtask completion.

Reviewer comment (Evaluation):
The evaluator currently is highly customized to BabyAI tasks.

Author response:
Thank you very much for this insightful question. The Evaluator's primary insight lies in providing guidance to both the Planner and the Actor through the design of intrinsic rewards. Compared to previous approaches, the Evaluator in PAE introduces two key innovations. Firstly, it leverages external knowledge to detect subtask completion, offering a more natural and comprehensible approach. Secondly, the Evaluator guides both the Planner and the Actor by encouraging the Planner to provide incremental guidance that is neither too easy nor too challenging, while motivating the Actor to complete tasks promptly under the Planner's guidance. **This design concept has broad applicability, and the Evaluator's key parameters are adaptive**, with a step threshold (t\*) that automatically grows as the agent's exploration ability improves, capped at 30% of the environment's maximum step count. To validate the generality of the Evaluator, we tested it in the MiniHack environment, where our PAE approach also exhibited superior performance ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP), [Screenshot of Addtional-Results](https://ibb.co/Wnsf0Qw)). Detailed experimental results are presented in Section 5 and Appendix A.2 in our revised version.

Reviewer comment (Experiments):
How did the authors reach the design for evaluator? What challenges if any, do the authors foresee in making the evaluator more general-purpose?

Author response:
Thank you very much for this insightful question. The Evaluator's primary insight lies in providing guidance to both the Planner and the Actor through the design of intrinsic rewards. Compared to previous approaches, the Evaluator in PAE introduces two key innovations. Firstly, it leverages external knowledge to detect subtask completion, offering a more natural and comprehensible approach. Secondly, the Evaluator guides both the Planner and the Actor by encouraging the Planner to provide incremental guidance that is neither too easy nor too challenging, while motivating the Actor to complete tasks promptly under the Planner's guidance. **This design concept has broad applicability, and the Evaluator's key parameters are adaptive**, with a step threshold (t\*) that automatically grows as the agent's exploration ability improves, capped at 30% of the environment's maximum step count. To validate the generality of the Evaluator, we tested it in the MiniHack environment, where our PAE approach also exhibited superior performance ([Screenshot of Training-curves](https://imgbb.com/QJKqdxP), [Screenshot of Addtional-Results](https://ibb.co/Wnsf0Qw)). Detailed experimental results are presented in Section 5 and Appendix A.2 in our revised version.