Paper:
Point clouds, as a form of Lagrangian representation, allow for powerful and flexible applications in a large number of computational disciplines. We propose a novel deep-learning method to learn stable and temporally coherent feature spaces for points clouds that change over time. We identify a set of inherent problems with these approaches: without knowledge of the time dimension, the inferred solutions can exhibit strong flickering, and easy solutions to suppress this flickering can result in undesirable local minima that manifest themselves as halo structures. We propose a novel temporal loss function that takes into account higher time derivatives of the point positions, and encourages mingling, i.e., to prevent the aforementioned halos. We combine these techniques in a super-resolution method with a truncation approach to flexibly adapt the size of the generated positions. We show that our method works for large, deforming point sets from different sources to demonstrate the flexibility of our approach.

Review:
The paper tries to learn temporally stable representations for point-based data sets and focus on varying size and dynamic point sets, and demonstrate its usefulness in the context of super-resolution. To deal with a difficult target that dynamically moves and deforms over time with variable input and output size, they take a novel temporal loss function for temporally coherent point set generation and siamese network setup for temporal loss calculation. Their novel temporal loss is based on EMD to minimize differences between an estimated point cloud and a desired super-resolution point cloud. The discussion and evolution on multiple loss functions are mostly well done. Except spatial loss is considered, taking the ground truth acceleration and estimated velocity into account is beneficial to this task. Their main contribution is taking permutation invariant loss terms and a siamese training setup and generator architecture, enabling improved output variance by allowing for dynamic adjustments of the output size, and identifying a specialized form of mode collapse for temporal point networks. 
They perform an empirical study of their temporal loss function on the generated data set and apply the proposed method to some complex 3D models to conclude the superior performance of temporal loss formulation in contrast to previous work.
 
Overall, this paper has some significant points on point cloud super-resolution, with the caveat for some clarifications on the theory and experiments. Given these clarifications in an author response, I would be willing to increase the score.
 
When the input moves slowly enough, the point cloud can be considered static. Can the proposed temporal loss outperform other works under this condition? 
 
Only one previous work PU-Net based on PointNet++ is compared in the paper, I would like to see more discussion on applying the proposed temporal loss with other point-based algorithms.
 
I am very curious about the effect on different choices of weighting terms hyperparameters in temporal loss and predefined upsampling factor r.