Paper: Directional Distance Field for Modeling the Difference between 3D Point Clouds (ICLR-2024)

Reviewer comment (Writing):
The name directional distance field is not suitable. I do not understand what is it until I finished reading the method section. The 'field' often indicates the signed distances or occupancies learned by a neural network. The proposed loss to measure distances from a reference point to the underline surface is not a 'field'.

Author response:
The field can also represent implicit fields calculated by classical methods, not limited to neural networks, such as IMLS [1], where Signed Distance Fields are estimated from point clouds using classical techniques without neural networks. Hence, utilizing 'Field' in our implicit representation of underlying surfaces is appropriate. \
The main idea of our distance metric is measuring the disparity between point clouds through the difference between the DDFs estimated from them, thus we denote our proposed DDF-based distance metric as 'DDF'.    
[1] Provably good moving least squares.

Reviewer comment (Writing):
The presentation can be improved. I suggest that the authors to improve the writings in the introduction to make the readers understand the loss more easily. The inappropriate name 'directional distance field' and the unexplained 'reference point' make the introduction not clear enough. I can not understand the loss until I finish reading the method section, but finally I find the loss quite simple.

Author response:
At the start of Section 3, we include a figure in the revised manuscript to visually depict the overall concept of our distance metric, enhancing clarity and comprehension.

Reviewer comment (Evaluation):
In Fig. 3, I find that DCD achieves quite good performances, why is the quantitative results of DCD in Tab. 1 that bad? More comparisons are needed.

Author response:
Table 1 of the manuscript shows the averaging numerical results of each category containing many 3D point clouds rather than only those displayed in the figure. In the updated manuscript, additional visual results have been included in Figure 11 within Appendix B. Furthermore, histograms depicting various evaluation metrics for different methods are presented in Figure 12, offering insights into the distribution of error and accuracy associated with each method.

Reviewer comment (Evaluation):
As shown in Fig.4, DDF is less efficient than CD and DCD. What is reason? Since CD measures point-to-point distances which should be much slower.

Author response:
In our metric, we introduce a set of reference points whose quantity exceeds the number of points in the point clouds. This allows us to indirectly establish correspondences between two point clouds.Calculating the DDFs at the position of each reference point involves K-NN searching, which constitutes the most time-consuming aspect of our method. Tables 9 and 10 in our manuscript illustrate the relationship between the running time and the number of reference points, as well as the KNN size. Unlike CD and DCD, which establish correspondence for each point in the point cloud by finding the 1-NN point in the other point cloud, our method operates slightly slower. However, when considering both efficiency and effectiveness, our metric outperforms CD, DCD, and EMD.

Reviewer comment (Experiments):
It will be interest to see the performance of DDC under more downstream tasks like point cloud completion, point cloud generation, etc.

Author response:
Thank you for your valuable suggestion. Due to time constraints, we have conducted a preliminary exploration of the application of our distance metric in the point cloud completion task. We utilized the widely used framework PCN [2], in which we substituted the CD with our distance metric. We maintained the same experimental settings as the original PCN and selected several categories from the ShapeNet dataset for our experiments, namely Airplane, Car, Table, and Chair.
The following table lists the numerical results, where it can be seen that the results from the network trained through our distance metric achieve higher F-Score than those from the network trained by CD.


|Airplane| F-Score (0.01)|
| ------ |  ------ | 
| CD | 0.8696 |
| Ours | 0.8725 |

|Car| F-Score (0.01)|
| ------ | ------ | 
| CD   |0.7238 |
| Ours  |0.7274 |

|Sofa|F-Score (0.01)|
| ------ | ------  | 
| CD | 0.5006 |
| Ours | 0.5135 |

|Table |F-Score (0.01)|
| ------ | ------ | 
| CD | 0.6910 |
| Ours | 0.7025 |

[2] PCN: Point Completion Network.

Reviewer comment (Writing):
The generation of reference points are not explained very clearly. The reviewer is confused by the shared identical weight operation and the reference point generation process.

Author response:
As explained in Section 3.2, the reference points serve to assess the disparity between the DDFs of the point clouds and are distributed in proximity to the implicit surfaces. Since each point in the point cloud is located on its underlying surface, we introduce offsets of Gaussian noise to displace these points away from the underlying surface while keeping them in close proximity. The standard deviations for the Gaussian noise are adjusted based on their distances to the nearest points in the point cloud, considering the non-uniformity of the point cloud. In the updated pdf file (i.e., the revised manuscript), we added one more figure, i.e., Fig. 3, to visually illustrate the generation process to help understanding.

Reviewer comment (Reproducibility):
The reference points are generated from one of the two point clouds, and the weights in Equ.1 are computed for each kNN points and reference points. The reviewer wants to ask how to share the weights in g(q_m, P1) and g(q_m, P2)?

Author response:
As stated in Section 3.3 of the manuscript, the points within $\Omega(\mathbf{q},\mathbf{P}_1)$ and $\Omega(\mathbf{q},\mathbf{P}_2)$ are first sorted based on their distances to the reference point $\mathbf{q}$, i.e., $||\mathbf{p}{\tiny 1,1}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny 1,K}-\mathbf{q}||$ with $\mathbf{p}{\tiny 1,1},...,\mathbf{p}{\tiny 1,K}\in\Omega(\mathbf{q},\mathbf{P}_1)$ and $||\mathbf{p}{\tiny 2,1}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny 2,K}-\mathbf{q}||$ with $\mathbf{p}{\tiny 2,1},...,\mathbf{p}{\tiny 2,K}\in\Omega(\mathbf{q},\mathbf{P}_2)$. This sorting process facilitates the transfer of weights from the target/ground-truth point cloud to the source/generated cloud. Specifically, for the calculation of directional distances across both point clouds, we adhere to the relation $w(\mathbf{q},\mathbf{p}{\tiny 2,k})=w(\mathbf{q},\mathbf{p}{\tiny 1,k}),\ k=1,...,K$. The weights can be computed from $\mathbf{P}_1$ (resp. $\mathbf{P}_2$) and then shared with $\mathbf{P}_2$ (resp. $\mathbf{P}_1$), according to the task.   
Our weight-sharing strategy is built upon the observation that if two point clouds are exactly the same, the weights obtained through the same calculation method should be equal for a typical reference point. If the weights are derived independently using the inverse distance for two point clouds, for the point cloud to be optimized/generated, the directional distance will be a high-order non-linear function of its points, thereby complicating the optimization process. The ablative study results in Table 7 of our manuscript show the superiority of the shared weights.

Reviewer comment (Experiments):
This work should add comparisons against relevant shape or surface descriptors in the literature. For example, against 3D shape context (and other methods) in Frome et al. ("Recognizing objects in range data using regional point descriptors." ECCV 2004). This lack of comparison is an important weakness as the proposed method resembles the common pipeline of: 1) Computing keypoints, here by using a sampling mechanism plus noise; 2) Obtaining descriptors at each keypoint, here as the concatenation of the magnitude and direction of a sum of weighted distances between a keypoint and the K-NN of a point cloud.

Author response:
**Response:** The reviewer may have **misunderstood** our Directional Distance Field (DDF) to some extent, resulting in the comparison with the mentioned surface descriptors not making sense. The detailed reasons are listed as follows. 
* The reviewer mentions the 3D Shape Context as a surface descriptor, which employs histograms to characterize local point cloud structures. However, this descriptor is **non-differentiable**, making it unsuitable for replacing our differentiable distance function  in the distance metric. Additionally, other differentiable descriptors, including learning-based ones, are **not viable** alternatives for the distance metric. This limitation essentially arises from these descriptors explicitly characterizing local surface structures, defined on surfaces where point clouds are sampled. In contrast, our DDF serves as an implicit field, representing underlying surfaces as its iso-surface with a specific value. Consequently, the DDF is defined across the entire 3D space, allowing it to be used at any point. 
* The key points used in the surface descriptor **differ** from the reference points used in our method. Key points, usually a subset of the whole point cloud, are selected to reduce computational complexity while retaining crucial information. And the correspondence between key points in different clouds is established through feature matching. Differently, the reference points in our method are located near the underlying surfaces and they are used to compute the difference between the DDFs of the point clouds, where the correspondence is established naturally according to these shared reference points.

Reviewer comment (Writing):
Another weakness is in formulations that should be presented more clearly, perhaps improving notation. For example, it is unclear why "g(qm, P1) and g(qm, P2) share identical weights". From equations, they seem to be different as calculated over different point clouds.

Author response:
**Response:** As stated in Sec. 3.3, the points within $\Omega(\mathbf{q},\mathbf{P}_1)$ and $\Omega(\mathbf{q},\mathbf{P}_2)$ are first sorted based on their distances to the reference point $\mathbf{q}$, i.e., $||\mathbf{p}{\tiny{1,1}}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny{1,K}}-\mathbf{q}||$ with $\mathbf{p}{\tiny{1,1}},...,\mathbf{p}{\tiny{1,K}}\in\Omega(\mathbf{q},\mathbf{P}_1)$ and $||\mathbf{p}{\tiny{2,1}}-\mathbf{q}||\leq...\leq||\mathbf{p}{\tiny{2,K}}-\mathbf{q}||$ with $\mathbf{p}{\tiny{2,1}},...,\mathbf{p}{\tiny{2,K}}\in\Omega(\mathbf{q},\mathbf{P}_2)$. This sorting process facilitates the transfer of weights from the target/ground-truth point cloud to the source/generated cloud. Specifically, for the calculation of directional distances across both point clouds, we adhere to the relation $w(\mathbf{q},\mathbf{p}{\tiny{2,k}})=w(\mathbf{q},\mathbf{p}{\tiny{1,k}}),\ k=1,...,K$. The weights can be computed from $\mathbf{P}_1$ (resp. $\mathbf{P}_2$) and then shared with $\mathbf{P}_2$ (resp. $\mathbf{P}_1$), according to the task.   
Our weight-sharing strategy is built upon the observation that if two point clouds are exactly the same, the weights obtained through the same calculation method should be equal for a typical reference point. If the weights are derived independently using the inverse distance for two point clouds, for the point cloud to be optimized/generated, the directional distance will be a high-order non-linear function of its points, thereby complicating the optimization process. The experimental results in Table 7 of our manuscript show the superiority of the shared weights.   
Eqs. (1) and (2) represent the calculation of our designed DDF. In Section 3.4 of the updated pdf file, we clarified the motivation of the shared weights.

Reviewer comment (Theory):
In Sec. 4.2, R should be in SO(3).

Author response:
**Response:**  In Section 4.2, we stated that '$\mathbf{R}\in\mathbb{R}^{3\times 3}$ is the rotation matrix', and this is equivalent to '$\mathbf{R}\in\texttt{SO}(3)$'. As the reviewer rightly noted, the latter representation is more concise. We updated the manuscript with this form.  
Thanks to the reviewer for the reminder. We have identified an inaccurate expression in the matrix multiplication $\mathbf{R}\mathbf{P}{\tiny src}$ in Eq. (5) of the manuscript, where $\mathbf{R}\in\mathbb{R}^{3\times 3}$ and $\mathbf{P}{\tiny src}\in\mathbb{R}^{N_{\small src}\times 3}$. To rectify this, we introduce a new symbol to denote the rigid transformation on the point cloud in Eq. (5) of the revised manuscript, denoted as $\mathcal{T}(\mathbf{P}{\tiny src},\mathbf{R},\mathbf{t})$.

Reviewer comment (Writing):
There is some abuse of notation when applying a rotation over a point cloud.

Author response:
**Response:**  In Section 4.2, we stated that '$\mathbf{R}\in\mathbb{R}^{3\times 3}$ is the rotation matrix', and this is equivalent to '$\mathbf{R}\in\texttt{SO}(3)$'. As the reviewer rightly noted, the latter representation is more concise. We updated the manuscript with this form.  
Thanks to the reviewer for the reminder. We have identified an inaccurate expression in the matrix multiplication $\mathbf{R}\mathbf{P}{\tiny src}$ in Eq. (5) of the manuscript, where $\mathbf{R}\in\mathbb{R}^{3\times 3}$ and $\mathbf{P}{\tiny src}\in\mathbb{R}^{N_{\small src}\times 3}$. To rectify this, we introduce a new symbol to denote the rigid transformation on the point cloud in Eq. (5) of the revised manuscript, denoted as $\mathcal{T}(\mathbf{P}{\tiny src},\mathbf{R},\mathbf{t})$.

Reviewer comment (Reproducibility):
The registration methods Opt-EMD, Opt-CD, Opt-ARL and Opt-Ours require better explanation.

Author response:
**Response:** In the figure, 'Opt-X' is an abbreviation for the Optimized Method based on Eq. (5), where a typical distance metric is used, i.e., 'X' can be CD, EMD, ARL, or our DDF. We have clarified this issue in the updated pdf file.