Paper: ITPNet: Towards Instantaneous Trajectory Prediction for Autonomous Driving (ICLR-2024)

Reviewer comment (Experiments):
The ablation study result was incomplete in the original submission, but I am good with the additional results provided in the rebuttal.

Author response:
Thanks for your advice. Due to the deadline, we directly set $N=10$ as the length of predicted unobserved locations without tuning. Following your suggestion, we conducted experiments with more $N$. The experimental results are listed in the table below. We will add them in the final version.

|  |           | | K=6| | | |K=6 | |
|- |-          |-|-|-|-|-|-|-|
| N| NRRFormer | minADE |minFDE | minMR| NRRFormer| minADE| minFDE| minMR|
|0 |    ×      |1.068|1.678|0.241|-|-|-|-|
|1 |    ×      |0.969|1.494|0.193|√|0.964|1.498|0.194|
|2 |    ×      |0.872|1.329|0.160|√|0.868|1.323|0.158|
|3 |    ×      |0.832|1.262|0.149|√|0.828|1.254|0.147|
|4 |    ×      |0.845|1.291|0.154|√|0.824|1.240|0.146|
|5 |    ×      |0.859|1.312|0.156|√|0.822|1.232|0.145|
|6 |    ×      |0.867|1.302|0.161|√|0.823|1.231|0.145|
|7 |    ×      |0.881|1.375|0.173|√|0.820|1.222|0.143|
|8 |    ×      |0.903|1.410|0.181|√|0.821|1.222|0.142|
|9 |    ×      |0.933|1.453|0.187|√|0.819|1.220|0.142|
|10|    ×      |0.967|1.522|0.196|√|0.819|1.218|0.141|


Based on the above table, we have some interesting findings: 
When NRRFormer is not enabled, the prediction error initially decreases and then increases with the increase of $N$. This is attributed to the introduction of noise and redundancy when predicting a longer feature sequence. When NRRFormer is enabled, the performance of our method is consistently improved as $N$ increases. This illustrates our NRRFormer model indeed can filter out redundant and noisy information, demonstrating its effectiveness.

Reviewer comment (Experiments):
Is ITPNet able to able to adaptively adjust the length of history used for different agents?

Author response:
Thanks for raising this concern which helps us to clarify the problem setting. We guess this concern is caused by the misunderstanding on the length of the history trajectories. We would like to clarify that ITPNet is actually able to adaptively adjust the length of history used for different agents. This adaptability is achieved through the utilization of both LSTM in the backward forecasting module and the self-attention mechanism in the NRRFormer module. These components enables our method to handle the input historical trajectories with varying lengths.

Besides, please kindly note that our problem setting is the same as that of MOE [1], the first work in instantaneous trajectory prediction. Thus, for a fair comparison, we also focus on the most extreme scenario, where only 2 frames of locations can be observed.

[1] Human Trajectory Prediction with Momentary Observation. CVPR'22

Reviewer comment (Experiments):
Will N=3 with NRRFormer enabled yield better result?

Author response:
Thanks for your advice. Due to the deadline, we directly set $N=10$ as the length of predicted unobserved locations without tuning. Following your suggestion, we conducted experiments with more $N$. The experimental results are listed in the table below. We will add them in the final version.

|  |           | | K=6| | | |K=6 | |
|- |-          |-|-|-|-|-|-|-|
| N| NRRFormer | minADE |minFDE | minMR| NRRFormer| minADE| minFDE| minMR|
|0 |    ×      |1.068|1.678|0.241|-|-|-|-|
|1 |    ×      |0.969|1.494|0.193|√|0.964|1.498|0.194|
|2 |    ×      |0.872|1.329|0.160|√|0.868|1.323|0.158|
|3 |    ×      |0.832|1.262|0.149|√|0.828|1.254|0.147|
|4 |    ×      |0.845|1.291|0.154|√|0.824|1.240|0.146|
|5 |    ×      |0.859|1.312|0.156|√|0.822|1.232|0.145|
|6 |    ×      |0.867|1.302|0.161|√|0.823|1.231|0.145|
|7 |    ×      |0.881|1.375|0.173|√|0.820|1.222|0.143|
|8 |    ×      |0.903|1.410|0.181|√|0.821|1.222|0.142|
|9 |    ×      |0.933|1.453|0.187|√|0.819|1.220|0.142|
|10|    ×      |0.967|1.522|0.196|√|0.819|1.218|0.141|


Based on the above table, we have some interesting findings: 
When NRRFormer is not enabled, the prediction error initially decreases and then increases with the increase of $N$. This is attributed to the introduction of noise and redundancy when predicting a longer feature sequence. When NRRFormer is enabled, the performance of our method is consistently improved as $N$ increases. This illustrates our NRRFormer model indeed can filter out redundant and noisy information, demonstrating its effectiveness.

Reviewer comment (Theory):
Why does $\hat{v}^{unobs}_1$ become mean of $V^{obs}$ on page 5? Is this mean for i=1,2 and all agents?

Author response:
Sorry for confusing you. The Eq. (4) on Page 5 aims to predict unobserved trajectories $\hat{\mathbf{V}}^{unobs} = \\{\hat{v}\_0^{unobs}, \hat{v}\_{-1}^{unobs}, \cdots, \hat{v}\_{-N+1}^{unobs} \\}$ based on the observed trajectories $\mathbf{V}^{obs} = \\{v^{obs}\_1, v^{obs}\_2\\}$ through LSTM.  We use the symbol $\hat{v}_1^{unobs}=\textbf{Mean}(\mathbf{V}^{obs})$ as the initial input of LSTM. When $i$ is $0, -1, \cdots, -N+1$, we use Eq. (4) to calculate $\hat{v}^{unobs}\_{i}$. We will refine it in the final version.

Reviewer comment (Experiments):
In comparison experiments with MOE or Distill, they do not seem to use HiVT or LaneGCN as backbone. But isn’t MOE or Distill also plug-and-play? For example, Distill still seems to be able to applied on HiVT or LaneGCN while maintaining the encoder and decoder structure and doing knowledge distillation. It seems fair to compare with MOE or Distill using same backbones (HiVT and LaneGCN). This may be the critical part for the fairness of the main experimental result, so if this is clarified, I think I can keep my rating more confidently.

Author response:
Thanks for your comments. Based on Table 1, we observed that HiVT achieved better performance than LaneGCN on both Argoverse and nuScenes. Therefore, we used HiVT as the backbone of MOE and Distill for a fair comparison. The implementation details can be found in the Appendix A.6.

Reviewer comment (Experiments):
Trajectory prediction generally predicts multiple futures, not one future. The proposed backwardly prediction seems to predict only one past, but have you ever experimented when predicting multiple pasts?

Author response:
Thanks for your comment. In our work, we propose a feature reconstruction loss, in an effort to predict the features instead of waypoints for unobserved history trajectories. The reason is as follows: Our ITPNet first utilizes the Backward Forecast module to predict the features of unobserved history trajectories, and then combines them with those of observed trajectories for predicting future trajectories. Therefore,  we believe that achieving a high level of precision in predicting the features of unobserved historical trajectories will be more beneficial for future trajectory prediction than focusing on unobserved historical waypoints. Therefore, we opt to predict features rather than waypoints for the unobserved historical trajectories.

When predicting the features of unobserved history trajectories, we simply choose the single-modal forecasting strategy. We think it is interesting to explore multi-modal method for predicting the features of unobserved history trajectories. This exploration may involve crafting appropriate loss functions tailored for multi-modal feature prediction.

Reviewer comment (Evaluation):
The addition of the cts loss and NRRFormer in abalation study results (Table 2) appear to have very small / questionable performance gains. Given the small change, can the authors speak to the consistency of these results? Given the small change, were multiple network seeds or trials done and do the same improvements remain? I would have found it useful to report a confidence interval or variance over the results although perhaps it is not conventional in this area.

Author response:
Thanks for your advice. Following your suggestion,  we further run the experiments 4 trials with different random seeds, and perform the t-test to verify that the improvements of the proposed components are statistically significant. The experimental results are listed in the table below. We can see that almost all p-values are less than the predetermined significance level of $\alpha=0.05$. This suggests that the performance is significantly improved, i.e., all the components in our method are effective for the instantaneous trajectory prediction task. 

For showcasing the effectiveness of our NRRFormer in more detail, please kindly refer to the response to Q3 for Reviewer ptje.

|           |           |          |      |       | K=1  |       |     |       | 
|-          |-          |-         |-     |-      |-     |-      |-    | -     |
| $L_{rec}$ | $L_{ctx}$ | NRRFormer|minADE|p-value|minFDE|p-value|minMR|p-value|
|          |           |          |   $4.146±0.080$   |   -    |   $8.347±0.074$   |   -    |  $0.844±0.011$   |   -    |
| √          |       |  |    $2.667±0.056$       |     $8.9\times10^{-6}$     |   $5.801±0.054$   |   $9.3\times10^{-7}$    |  $0.764±0.002$   |   $2.1\times10^{-4}$    |  
| √         | √         |          |  $2.615±0.013$    |    $0.049$   |   $5.737±0.016$   |    $0.030$   |  $0.761±0.001$   |   $0.029$    |
| √         | √         |     √    |   $2.577±0.041$   |   $0.065$   |   $5.646±0.051$   |   $0.014$    |  $0.753±0.003$   |   $0.007$    |



|           |           |          |      |       | K=6  |       |     |       | 
|-          |-          |-         |-     |-      |-     |-      |-    | -     |
| $L_{rec}$ | $L_{ctx}$ | NRRFormer|minADE|p-value|minFDE|p-value|minMR|p-value|
|          |           |          |   $1.089±0.003$   |   -    |   $1.719±0.006$   |   -    |  $0.248±0.002$   |   -    |
| √          |       |  |    $0.843±0.003$       |     $2.4\times10^{-7}$     |   $1.293±0.007$   |   $5.0\times10^{-9}$    |  $0.155±0.002$   |   $1.9\times10^{-6}$    |  
| √         | √         |          |  $0.832±0.002$    |    $0.005$   |   $1.263±0.003$   |    $0.002$   |  $0.149±0.001$   |   $8.1\times10^{-4}$    |
| √         | √         |     √    |   $0.822±0.002$   |   $9.4\times10^{-4}$   |   $1.222±0.003$   |   $2.0\times10^{-4}$    |  $0.143±0.001$   |   $3.9\times10^{-4}$    |

Reviewer comment (Experiments):
It would have insightful to report how the NRRFormer affects performance for smaller values of N in Table 3. As it currently is, section 4.4 “Analysis of Different Lengths $N$” seems somewhat rushed with mentions of how the usage of the NRRFormer was done “without tuning it carefully”.

Author response:
Thanks for your advice. Due to the deadline, we directly set $N=10$ as the length of predicted unobserved locations without tuning. Following your suggestion, we conducted experiments with more $N$. The experimental results are listed in the table below. We will add them in the final version.

|  |           | | K=6| | | |K=6 | |
|- |-          |-|-|-|-|-|-|-|
| N| NRRFormer | minADE |minFDE | minMR| NRRFormer| minADE| minFDE| minMR|
|0 |    ×      |1.068|1.678|0.241|-|-|-|-|
|1 |    ×      |0.969|1.494|0.193|√|0.964|1.498|0.194|
|2 |    ×      |0.872|1.329|0.160|√|0.868|1.323|0.158|
|3 |    ×      |0.832|1.262|0.149|√|0.828|1.254|0.147|
|4 |    ×      |0.845|1.291|0.154|√|0.824|1.240|0.146|
|5 |    ×      |0.859|1.312|0.156|√|0.822|1.232|0.145|
|6 |    ×      |0.867|1.302|0.161|√|0.823|1.231|0.145|
|7 |    ×      |0.881|1.375|0.173|√|0.820|1.222|0.143|
|8 |    ×      |0.903|1.410|0.181|√|0.821|1.222|0.142|
|9 |    ×      |0.933|1.453|0.187|√|0.819|1.220|0.142|
|10|    ×      |0.967|1.522|0.196|√|0.819|1.218|0.141|

Based on the above table, we  have some interesting findings: When NRRFormer is not enabled, the prediction error initially decreases and then increases with the increase of $N$. This is attributed to the introduction of noise and redundancy when predicting a longer feature sequence. When NRRFormer is enabled, the performance of our method is consistently improved as $N$ increases. This illustrates our NRRFormer model indeed can filter out redundant and noisy information, demonstrating its effectiveness.

Reviewer comment (Experiments):
It would be useful to show how the method’s performance changes for different number of prior observed locations (T) since only 2 prior observations are considered in this work (Table 4). During practical usage, I would assume that we would want to use all available prior observed locations for future trajectory prediction and so the T value will change. The authors have shown that their method outperforms baselines at T=2 prior observations, but does this trend continue for higher values of T? Does the method improve performance at all values of T versus baselines or is there a point where it is a detriment. For example, given a test trajectory with T=10 prior observations, do we trust the author’s method over baselines?

Author response:
Thanks for your comments. We would like to clarify that our ITPNet can take as input observed trajectories with varying lengths. This is achieved through the utilization of both LSTM in the backward forecasting module and the self-attention mechanism in the NRRFormer module. Following your suggestion, we added the experiments with different lengths of observed trajectories $T$, and compared our method with the two instantaneous trajectory prediction baselines, MOE and Distill. Our approach continues to demonstrate superior performance across various values of $T$ compared to the baselines.

|         |  |         |      |  K=1 |     |      | K=6  |     |
|---------|--|---------|------|------|-----|------|------|-----|
|Dataset  |T | Method  |minADE|minFDE|minMR|minADE|minFDE|minMR|
|         |  | MOE     |  3.312    |   6.840   |  0.794   |   0.939   |  1.413    |  0.177   |
|         |2 | Distill |  3.251    |  6.638    |  0.771   |   0.968   |  1.502    |  0.185   |
|         |  | ITPNet  |  2.631    |  5.703    |  0.757   |   0.819   |   1.218    |  0.141   |
|         |  | MOE     |  2.562    |   5.607   |  0.776   |   0.784   |  1.221    |  0.134   |
|Argoverse|5 | Distill |  2.465    |  5.452    |  0.756   |   0.796   |  1.248    |  0.139   |
|         |  | ITPNet  |  2.410    |  5.257    |  0.738   |  0.748   |   1.132    |  0.122   |
|         |  | MOE     |  2.357  |  5.141    |  0.733   |  0.726    |  1.101    |  0.117   |
|         |10| Distill |   2.224    |  5.039    |  0.726   |  0.731    |  1.118    |  0.119   |
|         |  | ITPNet  | 2.190   |   4.792   |  0.716   |  0.718    |  1.088    |  0.113   |

Reviewer comment (Experiments):
Although not needed at test time, the method requires ground-truth positions of unobserved states for the backward forecasting reconstruction loss during training. Depending on the dataset collection procedure, these may be hard to obtain. Furthermore – from what I understand – the HiVT and LaneGCN baselines in Table 1 are only trained with 2 observed prior locations and so it could be argued that the proposed approach requires more labeled data (predicts additional timesteps of prior locations which requires ground-truth labels). Although, at least for the HiVT method, the authors assert that training on all historic prior locations actually decreases performance (Figure 1) and so the second part of this criticism may be a moot. Nonetheless, I wonder if the additional labeled data could be used by the baselines in some other way (for example, training with variable length sequences).

Author response:
Thanks for your comments. We agree that it might be expensive to obtain the ground-truth positions of agents. Nevertheless, please kindly note that while we necessitate ground-truth positions of unobserved trajectories during training, the overall count of ground-truth positions in our method is fewer than that of standard trajectory prediction methods (e.g., $T=20$ for Argoverse). 

Following your suggestion, we conducted another experiment to enhance all baselines by incorporating the unobserved trajectory locations as part of the supervised signal. Specifically, we integrated an additional decoder into the baseline to predict the locations of unobserved trajectories. The results presented in the table below continue to illustrate the superior performance of our method over all other instantaneous trajectory prediction methods.

|         |         |      |  K=1 |     |      | K=6  |     |
|---------|---------|------|------|-----|------|------|-----|
|Dataset  | Method  |minADE|minFDE|minMR|minADE|minFDE|minMR|
|         | MOE     |  3.248    |   6.715   |  0.783   |   0.927   |  1.391    |  0.174   |
|Argoverse| Distill |  3.212    |  6.529    |  0.765  |   0.961   |  1.488    |  0.184   |
|         | ITPNet+HiVT  |  2.631    |  5.703    |  0.757   |   0.819   |  1.218   |  0.141|

Reviewer comment (Experiments):
The approach assumes given 2d locations as prior observations instead of raw sensory input. For the problem cases that this work attempts to address (example: vehicle suddenly emerging behind obstruction), I would wonder how accurate these 2d locations may be given limited tracking timesteps. Noisy or inaccurate initial 2d poses may have negative downstream consequences when input into the authors’ method and thus reduce the reported performance gains that they assert in their results. From what I understand, the authors simply truncated longer fully observed trajectories to 2 observations and so the unique circumstances of the previous problem case may be ignored.

Author response:
Thanks for your comments.  We performed another experiment where we added Gaussian noise $\mathcal{N}(0,\sigma)$ to two observed locations. The results provided in the table below demonstrate that our method still achieves superior performance compared to MOE and Distill, when the observed locations exhibit noise.




|         |           |      |  K=1 |     |      | K=6  |     |
|---------|---------|------|------|-----|------|------|-----|
|Dataset  | Method  |minADE|minFDE|minMR|minADE|minFDE|minMR|
|         |MOE                       |3.312|6.840   | 0.794  | 0.939 | 1.413  | 0.177  |
|         |Distill                   |3.251|6.638   | 0.771  | 0.968 | 1.502  | 0.185  |
|Argoverse|ITPNet+HiVT               |2.631|5.703   | 0.757  | 0.819 | 1.218  | 0.141  |
|         |MOE($\sigma=0.1m$)        |3.426|7.114   | 0.836  | 1.002 | 1.549  | 0.202  |
|         |Distill($\sigma=0.1m$)    |3.374|6.982   | 0.822  | 1.046 | 1.616  | 0.213  |
|         |ITPNet+HiVT($\sigma=0.1m$)|2.938|6.424   | 0.792  | 0.909 | 1.347  | 0.165  |

Reviewer comment (Writing):
On page 2, the usage of “straightly” in “Let’s consider a scenario where a vehicle travels straightly …” is awkward. Perhaps simply replace with the word “straight”.

Author response:
Thanks for your advice.  We will revise it in the final version.

Reviewer comment (Experiments):
The baselines is too old. Though LaneGCN and HiVT are both classic works, they are far from state-of-art-performace. Open sourced works like QCNet, MTR might worth trying.

Author response:
Thanks for your advice. In this work, we focus on studying the instantaneous trajectory prediction problem. Thus, we mainly compare ITPNet with the two most closely related approaches for instantaneous trajectory prediction, i.e., MOE and distill. LaneGCN and HiVT are two standard trajectory prediction models which need sufficient observed trajectories. Our method is plug-and-play and utilizes LaneGCN and HiVT as our backbones, respectively. We present the results of  LaneGCN and HiVT In Table 1, aiming to demonstrate that standard trajectory prediction methods struggle to effectively handle instantaneous trajectory prediction scenarios. Nevertheless, we believe it is a promising idea to integrate our method with state-of-the-art models, such as QCNet and MTR. In principle, our method can be compatible with QCNet and MTR. Due to the deadline, we plan to explore this in our future work.

Reviewer comment (Experiments):
Limited usage. The instantaneous trajectory prediction is interesting. However, the proposed method brings lots of extra parameters and computations. Let's discuss an actual deployment problem: I observe that even ITPNet+HiVT < HiVT with 2s inputs and during your training, all parameters of HiVT are tuned without freezing. Thus, during actual deployment, the system should run an extra inference of the ITPNet+HiVT for those instantaneous objects while running the original HiVT for all the other fully observed agents. I am not sure whether worth it to double the inference for those instantaneous objects.

Author response:
Thanks for raising the concern which is helpful for clarifying the real-world deployment problem. We guess that this concern may arise from a misunderstanding of the inference process. We would like to clarify that our ITPNet performs the inference ONLY once, as our method can take as input observed trajectories with varying lengths. This is achieved through the utilization of both LSTM in the backward forecasting module and the self-attention mechanism in the NRRFormer module.
Therefore, we have only one ITPNet+backbone model and conduct predictions with varying lengths of available observed trajectories in the real-world deployment, as mentioned by Reviewer ptje. Due to the deadline, we will perform such an experiment in the final version. Nevertheless, we believe that our method can achieve better performance in principle, as the involvement of more observed trajectories enables a more precise prediction of unobserved historical trajectories.

Besides, please kindly note that the reason for fixing the number of observed points $T=2$ is that we follow the same problem setting as that of MOE [1], the first work in instantaneous trajectory prediction. MOE focuses on the most extreme scenario, where only 2 frames of locations can be observed. 
For a fair comparison, we also fix $T=2$.

[1] Human Trajectory Prediction with Momentary Observation. CVPR'22

Reviewer comment (Theory):
Why only single-mode forcasting for history instead of multi-mode like for future prediction?

Author response:
Thanks for your comment. In our work, we propose a feature reconstruction loss, in an effort to predict the features instead of waypoints for unobserved history trajectories. The reason is as follows: Our ITPNet first utilizes the Backward Forecast module to predict the features of unobserved history trajectories, and then combines them with those of observed trajectories for predicting future trajectories.  Therefore,  we believe that achieving a high level of precision in predicting the features of unobserved historical trajectories will be more beneficial for future trajectory prediction than focusing on unobserved historical waypoints. Therefore, we opt to predict features rather than waypoints for the unobserved historical trajectories.

When predicting the features of unobserved history trajectories, we simply choose the single-modal forecasting strategy. We think it is interesting to explore multi-modal method for predicting the features of unobserved history trajectories. This exploration may involve crafting appropriate loss functions tailored for multi-modal feature prediction.

Reviewer comment (Experiments):
Did you compare the results of predicting waypoints and predicting features?

Author response:
Thanks for your comment. In our work, we predict the features instead of waypoints for unobserved history trajectories. The reason is as follows: Our ITPNet first utilizes the Backward Forecast module to predict the features of unobserved history trajectories, and then combines them with those of observed trajectories for predicting future trajectories. Therefore,  we believe that achieving a high level of precision in predicting the features of unobserved historical trajectories will be more beneficial for future trajectory prediction than focusing on unobserved historical waypoints. Therefore, we opt to predict features rather than waypoints for the unobserved historical trajectories.
Nevertheless, it is worthwhile to conduct further investigation into whether predicting features yields superior results compared to predicting waypoints, considering both methodological and empirical perspectives. Due to deadline, we will explore it in our future work.

Reviewer comment (Experiments):
The contrastive loss seems have little influence. The authors might consider multiple tries.

Author response:
Thanks for your advice. Following your suggestion,  we further run the experiments 4 trials with different random seeds, and perform the t-test to verify that the improvements of the proposed components are statistically significant. The experimental results are listed in the table below. We can see that almost all p-values are less than the predetermined significance level of $\alpha=0.05$. This suggests that the performance is significantly improved, i.e., all the components in our method are effective for the instantaneous trajectory prediction task. 

For showcasing the effectiveness of our NRRFormer in more detail, please kindly refer to the response to Q3 for Reviewer ptje.

|           |           |          |      |       | K=1  |       |     |       | 
|-          |-          |-         |-     |-      |-     |-      |-    | -     |
| $L_{rec}$ | $L_{ctx}$ | NRRFormer|minADE|p-value|minFDE|p-value|minMR|p-value|
|          |           |          |   $4.146±0.080$   |   -    |   $8.347±0.074$   |   -    |  $0.844±0.011$   |   -    |
| √          |       |  |    $2.667±0.056$       |     $8.9\times10^{-6}$     |   $5.801±0.054$   |   $9.3\times10^{-7}$    |  $0.764±0.002$   |   $2.1\times10^{-4}$    |  
| √         | √         |          |  $2.615±0.013$    |    $0.049$   |   $5.737±0.016$   |    $0.030$   |  $0.761±0.001$   |   $0.029$    |
| √         | √         |     √    |   $2.577±0.041$   |   $0.065$   |   $5.646±0.051$   |   $0.014$    |  $0.753±0.003$   |   $0.007$    |



|           |           |          |      |       | K=6  |       |     |       | 
|-          |-          |-         |-     |-      |-     |-      |-    | -     |
| $L_{rec}$ | $L_{ctx}$ | NRRFormer|minADE|p-value|minFDE|p-value|minMR|p-value|
|          |           |          |   $1.089±0.003$   |   -    |   $1.719±0.006$   |   -    |  $0.248±0.002$   |   -    |
| √          |       |  |    $0.843±0.003$       |     $2.4\times10^{-7}$     |   $1.293±0.007$   |   $5.0\times10^{-9}$    |  $0.155±0.002$   |   $1.9\times10^{-6}$    |  
| √         | √         |          |  $0.832±0.002$    |    $0.005$   |   $1.263±0.003$   |    $0.002$   |  $0.149±0.001$   |   $8.1\times10^{-4}$    |
| √         | √         |     √    |   $0.822±0.002$   |   $9.4\times10^{-4}$   |   $1.222±0.003$   |   $2.0\times10^{-4}$    |  $0.143±0.001$   |   $3.9\times10^{-4}$    |