Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The paper tries to learn temporally stable representations for point-based data sets and focus on varying size and dynamic point sets, and demonstrate its usefulness in the context of super-resolution. To deal with a difficult target that dynamically moves and deforms over time with variable input and output size, they take a novel temporal loss function for temporally coherent point set generation and siamese network setup for temporal loss calculation. Their novel temporal loss is based on EMD to minimize differences between an estimated point cloud and a desired super-resolution point cloud. The discussion and evolution on multiple loss functions are mostly well done. Except spatial loss is considered, taking the ground truth acceleration and estimated velocity into account is beneficial to this task. Their main contribution is taking permutation invariant loss terms and a siamese training setup and generator architecture, enabling improved output variance by allowing for dynamic adjustments of the output size, and identifying a specialized form of mode collapse for temporal point networks. 
They perform an empirical study of their temporal loss function on the generated data set and apply the proposed method to some complex 3D models to conclude the superior performance of temporal loss formulation in contrast to previous work.
 
Overall, this paper has some significant points on point cloud super-resolution, with the caveat for some clarifications on the theory and experiments. Given these clarifications in an author response, I would be willing to increase the score.
 
When the input moves slowly enough, the point cloud can be considered static. Can the proposed temporal loss outperform other works under this condition? 
 
Only one previous work PU-Net based on PointNet++ is compared in the paper, I would like to see more discussion on applying the proposed temporal loss with other point-based algorithms.
 
I am very curious about the effect on different choices of weighting terms hyperparameters in temporal loss and predefined upsampling factor r.


review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: The paper addresses the task of learning temporally stable features for point clouds with an application to upsampling point clouds. Learning point-based descriptors has been a major topic of research in the recent vision and graphics meetings, where approaches have been proposed focusing semantic labeling, geometry-oriented tasks (e.g. normal estimation), and point-based graphics. However, as the paper states, and to the best of my knowledge, no methods have been proposed to learn features in fourth dimension in a temporally stable way. Thus, the very topic of research is significantly novel and promising. 

The authors consider a combination of loss functions and train a neural point-based network to learn the features. To stabilize training, the authors carefully study the effect of a series of loss functions, including well-known EMD, losses ensuring slow changes in positions, velocities, and accelerations, as well as a mingling loss to ensure a more uniform spatial point distribution on the output shape. The studied losses are very logical to implement as one aims to ensure that the output should satisfy a temporally smooth motion pattern. 

The experimental results provide a clear view of the proposed approach and demonstrate that combining the studied objectives function with a known point-based learning approach leads to a temporally stable feature representation per-point. An ablation study further helps to validate the proposed approaches step-by-step.

To sum up, I believe paper should clearly be accepted, as (1) the work addresses a novel point-based learning task, (2) the research methodology is convincingly presented, and (3) the results provide a clear demonstration of the feasibility of the proposed task.

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: Summary:
This paper proposed a deep network for point cloud sequence super-resolution/upsampling. Building on the basis of pointNet and PU-net, the main contribution of the paper is identifying the problem of temporal incoherence in the process of upsampling a point cloud shape representation as well as a training loss to encourage temporal coherence. In the cases showed in the paper, the proposed method seems effective comparing to previous work which is not done on sequence data. My main concern about the work is that the experimental evaluation is limited.

Strengths:
Interesting problem and novel idea.
The proposed method is technically sound. From the provided results, the newly introduced training loss seems effective: the result sequences are visually more plausible and smooth.
Weaknesses:
Qualitative results are limited and in most cases seemingly simple. In the paper as well as the companion video, there are very few examples provided. The scale of the evaluation demonstration is not convincing enough for the readers that this work could be generalized to more complicated testing scenarios.
Quantitative results are also limited. Since the method is handling the coherence of shape deformation over time, it would be much more convincing and helpful to introduce a dense-correspondence evaluation as a benchmark. For example, one can create ground-truth correspondence from parametric morphable models and evaluate the coherence of the sequence by comparing the generated results with the ground truth.

review_title: Official Blind Review #2

Meta-review: This paper provides an improved method for deep learning on point clouds.  Reviewers are unanimous that this paper is acceptable, and the AC concurs. 
Decision: Accept (Spotlight)