R4D-planes: Remapping Planes For Novel View Synthesis and Self-Supervised Decoupling of Monocular Videos

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 11:28:18

Abstract: The view synthesis and decoupling of dynamic objects from the static environment in monocular video are both long-standing challenges in CV and CG. Most of the previous NeRF-based methods rely on implicit representation, which require additional supervision and training time. Later, various explicit representations have been applied to the task of novel view synthesis for dynamic scenes, such as multi-planes or 3D gaussian splatting. They usually encode the dynamics by introducing an additional time dimension or a deformation field. These methods greatly reduce the time consumption, but still fail to achieve high rendering quality in some scenes, especially for some real scenes. For the latter decoupling problem, previous neural radiation field methods require frequent tuning of the relevant parameters for different scenes, which is very inconvenient for practical use. We consider above problems and propose a new representation of dynamic scenes based on tensor decomposition, which we call R4D-planes. The key to our method is remapping, which compensates for the shortcomings of the plane structure by fusing space-time information and remapping to new indexes. Furthermore, we implement a new decoupling structure, which can efficiently decouple dynamic and static scenes in a self-supervised manner. Experimental results show our method achieves better rendering quality and training efficiency in both view synthesis and decoupling tasks for monocular scenes.

Main paper: 
 related work 2.1 explicit representations for dynamic scenes: 
 Explicit representations have been widely applied to dynamic scenes, voxel is an effective way to do this. Tineuvox [5] uses a deformation net and replaces the MLP for static canonical space with a voxel grid. NeRFPlayer [24] shows that the time dimension cannot simply be introduced, which would make the memory consumption of long videos unaffordable. Temporal interpolation NeRF [20] suggests modeling different time periods separately and then performing temporal interpolation to get the results, but this does not seem to work for all scenarios. The VM decomposition in TensoRF [4] makes voxel representation much less consuming, and K-planes, Hexplane, and Tensor4D [3, 6, 23] decompose dynamic scenes based on this low-rank decomposition with 2D tensors. HumanRF [11] defines 4D feature grids as a decomposition of four 3D and four 1D feature grids and uses adaptive temporal partitioning to model human motion. However, without additional supervision, for monocular video, especially in real-world scenes, although these decomposition methods can cope with some topology changes and reduce the time overhead, the results are not comparable to previous methods, thus they are more likely to be used in multi-view settings. Recently, due to the excellent performance of 3DGS [12] , some recent work has extended it to dynamic scenes as well. Similar to NeRFs, some replace the 3D gaussians with 4D gaussians and others introduce a deformation field. However, current works seem to have difficulty in modeling large motions and rapid scene changes.To reduce the complexity of dynamic scenes, there are already some approaches that tend to model dynamic objects and static backgrounds separately, but they aim to improve the quality of the rendering rather than cleanly decoupling the static and dynamic parts at the same time [14] . STaR [33] proposes a framework for selfsupervised decoupling and reconstruction of moving objects and scenes, but it is only suitable for a single rigidly moving object and requires multi-view videos. Using HyperNeRF [19] and NeRF [16] to model the dynamic foreground and static background respectively, D 2 NeRF [28] achieves self-supervised decoupling of dynamic and static objects and clean rendering results of both through a series of loss functions and hyperparameters. However, apart from the time it takes, the noise in static rendering caused by the occlusion of dynamic objects cannot be ignored. In addition, frequent tuning of the hyper-parameters for different scenes definitely makes the application more difficult. SUDS [26] advantages the method in D 2 NeRF and applies it to large-scale scenarios, building a largescale dynamic NeRF supervised by signals such as radar, optical flow, etc., and enabling downstream tasks such as viewpoint synthesis, 3D scene flow estimation, etc. RobustNeRF [22] models distractors in training data as outliers of an optimization problem, and removes outliers from the scene to get a clean static rendering result, but the distractor is the same or different target in frames, rather than an object moving continuously over time. 
method: 
 In this section, we first review the general architecture of multiplane representations(Sec.3.1). We then describe the implementation of the remapping method and illustrate the representation for view synthesis task(Sec.3.2). We show the newly proposed structure for the dynamic and static decoupling task, which is based on tensor decomposition(Sec.3.3). Finally, we show the optimization details in experiments(Sec.3.4).Most of the neural radiation fields based on explicit representations employ the form of hybrid neural fields i.e., for sampled points in space, interpolation is performed at the corresponding positions in the explicit representation to obtain the latent features, then they will be decoded by a small MLP to obtain the attributes of the radiance field, color or density. This hybrid structure gives a unique advantage to the neural radiance field, achieving good performance in terms of quality and efficiency of rendering.The multi-plane representation also follows the hybrid structure described above, with a decomposition of the tensor representing the latent feature volume in it. The original dense grid is decomposed into a series of low-dimensional factors, which greatly reduces memory consumption and makes the structure more compact and efficient. The sampled points in the space are projected into the corresponding low-dimensional coordinate space, and the latent features are obtained by interpolating over the projected point positions,then feature aggregation such as concatenation, multiplication, or addition will be performed to obtain the actual latent features 𝐷 . The process can be expressed as:EQUATION)where Aggr denotes the aggregation process, Proj denotes projection, Interp denotes interpolation, 𝑝 is the position of the sampled points, 𝐹 1 , ...𝐹 𝐹 denotes the corresponding low-dimensional factors.For the 4D representation of a dynamic scene, it is usually decomposed into the form of Hexplane [3] . It uses six learnable parameter planes to encode temporal and spatial information. For a sampled point (𝑥, 𝑦, 𝑧, 𝑡) in space at a given moment, it will be projected onto the six planes including XY-ZT, YZ-XT, XZ-YT and the feature vectors of the point will be computed by the following form:EQUATIONwhere 𝑀 𝐴𝐵 𝑟 ∈ R 𝐴𝐵 is the feature in the corresponding plane after interpolation, 𝐷 is the feature vector that will be decoded.In this section, we present our framework for dynamic monocular reconstruction. Previous representation based on tensor decomposition, such as Hexplane [3] , have shown limitations in monocular settings. The structure lacks an information sharing mechanism between frames, cannot exploit the information in different frames to constrain the object reconstruction. On the other hand, due to lack of constraints in monocular dynamic scenes, optimizing the tensor to be low-rank is hard. In fact, in dynamic scenes, the color and density can change abruptly in space and time, Total Variational (TV) loss cannot effectively impose constraints in this case, instead it makes neighboring feature in the structure similar. Furthermore, representations like voxel or tri-plane using 𝑥𝑦𝑧𝑡 coordinates are difficult to capture high-frequency signals due to interpolation at a limited resolution. Previous work use multi-resolution in space to alleviate this problem, but our work takes a different approach.As shown in Fig. 1 , when a point 𝑝 in a ray with direction (𝜃, 𝜙) is sampled at (𝑥, 𝑦, 𝑧, 𝑡). The time t will be first encoded by a small MLP F and then concatenated with the position encoding of (𝑥, 𝑦, 𝑧) into the remapping MLP G to obtain the new coordinate (𝑟 𝑥 , 𝑟 𝑦 , 𝑟 𝑧 , 𝜏). Among them,𝜏 is activated by 𝑡𝑎𝑛ℎ, compressing its range to [-1,1]:EQUATIONwhere 𝛾 is the position encoding:𝛾 (𝑥) = (𝑠𝑖𝑛(2 0 𝑥), 𝑐𝑜𝑠 (2 0 𝑥), ..., 𝑠𝑖𝑛(2 𝐿-1 𝑥), 𝑐𝑜𝑠 (2 𝐿-1 𝑥)) (4)where 𝐿 is a hyperparameter which controls frequency of the encoding.Remapping has two main functions: firstly, it fuses information from time and space, introducing a mechanism for sharing information across frames. Secondly, the interpolation position is also optimized compared to direct indexing using fixed 𝑥𝑦𝑧𝑡 coordinate, which means that the original samples are reconfigured and have different scales in the new dimension.Then we project it onto these six planes and use linear interpolation to get the density feature: The structure for decoupling of static and dynamic task. We are the first to decouple the static and dynamic parts for monocular video in a self-supervised manner using explicit representation. We use the remapping representation as the dynamic component, and for the static component, we follow the vm decomposition in TensoRF [4] to represent the static space, since it does not include the time dimension. into color and density by a direct sum or a tiny MLP. Is remapping block a deformation field? Although we use an MLP to output a set of values about coordinates, the idea of remapping has no relation to deformation fields. For the method of reconstructing dynamic scenes using a deformation field, the key is to map the sampled point 𝑝 given a moment(𝑝 ∈ R 4 ) into a canonical space 𝐻 (𝐻 ∈ R 3 ), compressing the originally four-dimensional space into a three-dimensional canonical space. This assumes that any point in the originally four-dimensional space must have a counterpart in the canonical space. But for some cases this assumption is not correct.D =P 𝑅 𝑥 𝑅 𝑦 𝐹 𝜎,𝑟 𝑥 𝑟 𝑦 • P 𝑅 𝑧 𝑇In contrast, our method is a remapping of the position, a transformation of the entire domain, a mapping from four dimensions to four dimensions, with no loss of information in scenes, and experiments have demonstrated that our method is effective in capturing topological changes, which is difficult for deformation methods.In the case of dynamic-static decomposition, we follow the above structure as our dynamic representation, the structure is shown in Fig. 2 . For the static background we use the VM decomposition in TensoRF [4] and set the dimensions of vector and matrix to the same as the dynamic one:EQUATIONandEQUATIONStatic and dynamic components have their own decoders S 𝑠𝑡 and S 𝑑 𝑦 :EQUATIONFor dynamic component:EQUATIONthen the pixel colour Ĉ at a given frame is calculated by volume rendering:EQUATIONwhereEQUATION)EQUATIONandEQUATIONIn this setup, dynamic and static scenes can naturally be rendered individually, following the body rendering equations:Ĉ𝑑𝑦 (r) = 𝑁 ∑︁ 𝑖=1 T 𝑑 𝑦 𝑖 𝛼 𝑑𝑦 𝑖 𝑐 𝑑𝑦 𝑖 (15)whereEQUATION)andEQUATION)where T 𝑠𝑡 𝑖 = exp - 𝑖 -1 ∑︁ 𝑗=1 𝜎 𝑠𝑡 𝑗 𝛿 𝑗 (18)Loss. For monocular view synthesis, the optimization objective is:EQUATION)EQUATION)EQUATION)where r is the sampled ray, 𝑖, 𝑗 are indices on the plane's resolution. In the conditions of this task, we only set the MSE loss and the TV loss without adding any additional regularization in order to demonstrate the robustness of our method.For the self-supervised decoupling of static and dynamic, the loss is:EQUATION)EQUATION)where𝐻 (𝑥) = -(𝑥 • log(𝑥) + (1 -𝑥) • log(1 -𝑥)) L 𝑑 (r) = max 𝜎 𝑑 𝑦 𝜎 𝑠𝑡 + 𝜎 𝑑 𝑦 (24)L 𝑑 and L 𝑠 are losses that help decouple scenes in D 2 NeRF [28] . In fact, it can be seen in the framework that for the dynamic component part of it, the representation of the scene is higher than for the static part. Indeed, for the decoupling task in neural radiance field, dynamic components will inevitably have greater expressive power than static components.This can lead to the fact that during decoupling, dynamic components tend to incorrectly decouple the static in the scene as dynamic. In D 2 NeRF [28] the authors control the shift of the extremes of L 𝑠 by adjusting the hyperparameter 𝑘. Thus, the dynamic part is suppressed and the density is preferentially decoupled to static. However, this parameter is not deterministic and needs to be readjusted for different scenarios to determine an optimal value, which makes the application of decoupling impractical.Our goal is to achieve a more robust decoupling of the radiance field, so we no longer rely on the hyperparameter 𝑘. In all the experiments of the scenarios, we set it uniformly to 1.5. Instead we propose a new optimization strategy, called different low-density initialization. Different low-density initialization. For the decoupling settings, we propose to use different low-density initializations for static and dynamic components. Specifically, we use the 𝑠𝑜 𝑓 𝑡𝑝𝑙𝑢𝑠 function to activate the density values, and add an additional offset 𝑏, for 𝜎 = softplus(𝜎 + 𝑏), we set the 𝑏 in the dynamic component much lower than the static, which can help to better decouple the dynamic from the static, and alleviate the dependence of self-supervision method on hyperparameter tuning. Coarse to fine training. A coarse-to-fine sampling scheme is also used. Unlike [3, 5, 6] , since we use the new four-dimensional coordinates after remapping, we progressively increase the resolution of all four dimensions. During training, we keep a mask to record low-density points in spacetime and skip these samples. Feature decoder. For synthetic scenes, we use MLP as the density decoder; for real scenes, we directly sum the features to obtain the density. In both scenes, we use a small MLP as the appearance decoder. Windowed positional encoding. We change the position coding to windowed position coding proposed in [19] in the decoupling task for real scenes. The window function is:EQUATIONwhere 𝑗 ∈ {0, . . . , 𝑚 -1} is the index of the frequency band, and 𝛼 ∈ [0, 𝑚] is linearly increasing. We use it for encoding the spatial coordinates before remapping and the appearance feature in the task of decoupling. 
experiments: 
 We experimented with our method on view synthesis and decoupling tasks in the monocular dataset. All experiments are implemented with PyTorch on an RTX 3090 GPU. The learning rate for feature planes is 0.02, and the learning rate for neural networks is 0.001 except for remapping net, which is 0.0007. All learning rates are exponentially decayed and kept in the same settings in all experiments. Please see supplementary material for visual comparisons of ablation experiments, detailed metrics and renderings. For novel view synthesis, we used the 360°D-NeRF [21] dataset for synthetic scenes and the HyperNeRF [19] dataset for realistic scenes, respectively. D-NeRF dataset is a monocular synthetic 360°dataset containing eight different dynamic scenes, each under large deformations and realistic non-Lambertian materials. The training set consists of 50 to 200 frames and the test set consists of 20 frames, each with a precise timing record.We set 𝐹 = 48 for the appearance plane and 𝐹 = 24 for the density plane,starts with 64 3 for space resolution and 0.25 × frames for time resolution and upsamples at 3k, 6k, 9k to 200 3 and 0.5 × frames,which is consistent with [3] . We show our results in Table 1 , the results show that our method achieves better results in each metric compared to previous SOTA methods using this dataset including NDVG, V4D, Tineuvox, Hexplane, Kplanes, D-NeRF and Temporal interpolation NeRF [3, 5-7, 9, 20, 21] , and most recent method 4DGS [27] , GS4D [31] .See Supplementary Material for more detailed results and settings.HyperNeRF dataset contains video sequences from mobile phones. There are four sequences of validation rig scenes that were recorded using two vertically aligned mobile phones. Unlike the D-NeRF dataset, the HyperNeRF dataset does not have a precise time record; it uses the sequential ids of the frames as time information. We set 𝐹 = 24 for the appearance plane and 𝐹 = 8 for the density plane, use summation instead of MLP to compute the density, and upsample resolution at 2k, 4k, 6k, and 8k. We show our results in Table 2 , and please see Fig. 4 for a visual comparison. We show in detail the relevant metrics and rendering pictures, which show that we have achieved better results. In addition we show the limitations of the current dynamic representation methods based on Gaussian splatting, which include deformable 3DGS [30] as well as 4DGS [27] , these methods are not able to model rapid motions in the scene, such as the material columns of the printer, while our method can accurately reconstruct them. Please see more examples in supplementary material. It is worth mentioning that deformable 3DGS attributes this to the inaccuracy of the camera poses of the HyperNeRF dataset, this claim is inaccurate, since the static scene is reconstructed. Note that we do not and cannot use the deformation PSNR ↑ SSIM ↑ LPIPS ↓ Tineuvox [5] 32.67 0.97 0.04 NDVG [9] 31.31 0.97 0.046 V4D [7] 33.72 0.98 0.02 Hexplane [3] 31.04 0.97 0.04 K-planes-explicit [6] 30.39 0.96 -K-planes-hybrid [6] 30.84 0.96 -D-NeRF [21] 30.50 0.95 0.07 T-interpolation NeRF [20] 32.73 0.97 0.03 4DGS [27] 34.05 0.98 0.02 GS4D [31] 34 We followed the dataset in D 2 NeRF [28] , including some scenes in HyperNeRF [19] and new scenes. In keeping with the view synthesis settings, we test all of the vrig(validation rig) datasets include eight challenging real scenes. The corresponding experimental settings are consistent with view synthesis settings in real scenes. Due to the lack of relevant metrics to measure the effect of decoupling, we show the metrics for the synthetic result as well as a comparative picture of decoupling. The results show that our method takes the lead in both final synthesis metrics and training efficiency. Although we set the same hyperparameters for all scenes, we achieve comparable decoupling results,which proves the effectiveness of our strategies. In addition, the static backgrounds produced by D 2 NeRF always have noise in the occluded part of the dynamic object, which is not present in our method. We performed ablation experiments on the remapping block on the D-NeRF dataset [21] . see Table 1 , and please see Fig. 6 for a visual comparison. The plane structure without remapping suffers from serious shortcomings in some monocular scenes.For the decoupling task, we performed ablation experiments on the D 2 NeRF dataset for the initialization strategy proposed in this paper. Without tuning the hyperparameters in the loss, our proposed initialization strategy significantly improves the effectiveness of decoupling. In contrast, the radiance field without initialization cannot be effectively decoupled in a self-supervised manner. 
limitation: 
 Although our proposed remapping plane achieves good results in the tasks of novel view synthesis as well as dynamic and static decoupling, it still has shortcomings. In fact, since our method still follows volume rendering, compared to the current state-of-the-art 3D Gaussian splatting, for rendering speed, the 3D Gaussian splatting is much faster than the methods based on volume rendering,  
conclusion: 
 We propose R4D-plane, which addresses the shortcomings of the plane structure by using the new 4D domain and its coordinates after remapping. In addition to view synthesis task, we extend it to dynamic-static decoupling. Experimental results of both tasks show the effectiveness of our approach, it significantly improves the rendering quality in monocular scenes and achieves faster and better dynamic-static decoupling compared to the previous approach. We hope that R4D-planes can contribute to the subsequent related work.  
: 
 demonstrating impressive performance in both reconstruction accuracy and rendering quality with only multi-view images and corresponding camera position information. However, it cannot be ignored that data in the form of dynamic monocular video is the most common and easily accessible form for a variety of real-life situations with practical capture conditions.Many current approaches for dynamic scenarios extend on the basis of static implicit representations. However, implicit representations rely on large MLPs [13, 14, 18, 19, 21, 29, 33] , which typically incur significant training and inference time. At the same time, the nature of MLP causes the radiance field to be unable to effectively capture the high frequency signals in the scene.In order to improve the rendering efficiency of NeRF, researchers have proposed various improvement strategies, such as improved sampling strategies or pre-computation . Later, a number of different explicit representations have been explored, including voxels [2, 17, 25] , low-rank tensor decomposition [4, 10] , 3D Gaussians [12] , etc. These representations lead to breakthroughs in rendering efficiency, some even enabling real-time rendering. In terms of quality improvement, Mip-NeRF effectively reduces aliasing artifacts and dramatically improves the ability to express fine details, its core idea has subsequently been applied in related explicit representations, including voxels [2] , tri-planes [10] , and 3D Gaussians.These representations are quickly extended to dynamic scenes, unlike implicit representations, explicit structures usually greatly reduce time consumption, but increase memory overhead. Most of them use hybrid methods [3, 5, 6, 9] , which typically extract features using explicit structures and decode them using a small MLP. Representations in these methods are usually in the form of voxel mesh or its decomposition, which still presents some inconveniences when dealing with dynamic scenes. For example, simply extending the voxel grid to four dimensions will lead unaffordable memory consumption, while this problem can be alleviated by employing decomposed representations, tensor decomposition structures need to be low-rank, which are often difficult to optimize due to lack of constraints in dynamic monocular scenes. Another methods follow the pipeline of 3D Gaussian Splatting(3DGS), using 4D gaussians [31] or deformable 3D gaussians [27, 30] to represent the dynamics, but cases show they fail to modeling large motions and rapid scene changes. Therefore, there are still some problems in monocular scenes. Using additional supervision, such as optical flow [14, 15] , depth [1, 14, 15] , etc. would be of great help, but in practical situations those are often not easily obtained.We consider the above problems and make further improvements to the multi-plane representation based on tensor decomposition. In general, the tensor decomposition structure is sparse and low-rank. However, optimizing such a low-rank grid is not straightforward, studies [8, 32] have shown that optimization can be stable only when the frame of representation aligns with the scene or signal structure. On the other hand, for data in the form of monocular videos, there must be an information interaction mechanism between the video frames to maintain the consistency of geometry. Previous multi-plane methods [3, 6] ignored above factors, resulting in poor rendering quality and geometry in monocular scenes.In this paper, we propose a new representation of dynamic scenes based on tensor decomposition. We focus on exploiting spatial location and temporal information by remapping the 4D coordinates through the remapping net, which allows for a better fusion of the spatio-temporal information with the explicit planes. By remapping, we achieve a separation of the scene from the explicit representation (position of sampling point in the scene is not same as explicit representation), which makes the alignment condition non-essential and reduces the difficulty of optimization. Except for the task of view synthesis, we also designed a new structure for the self-supervised decoupling task, greatly reducing the time cost, and achieving promising performance in decoupling and synthesis results.In summary, the main contributions of this work are as follows:• A new representation of dynamic scenes, which extracts the feature by the new 4D coordinates after remapping, can overcome the shortcomings of previous methods and achieve realistic rendering results.• A new explicit structure for dynamic and static self-supervised decoupling and its optimization strategies, showing better efficiency and decoupling results.• Experiments show that our method is simple and effective, bridges the gap of previous multi-plane methods on monocular data, and achieves high-quality results on multiple datasets without additional supervision. 
 
 Appendix:  


Decision: All the reviews were positive originally. The reviewers found that this paper is well written, the idea is sound and the improvement is good to demonstrate the impact of the remapping function when using Nd-Plan framework. After the rebuttal, one of the reviewers lowered the score from borderline acceptance to borderline reject due to the concern with the lack of significant advantage in computational cost, the presentation in Table 2 and the insufficient evidence of good motion reconstruction quality. Given the overall positive reviews and most of the concerns raised by reviewer K7eN can be addressed in the revision, we'd like to recommend the acceptance of this paper. We would encourage the authors to carefully revise the paper to incorporate reviews to improve the paper.

Reviews: review: paper_summary: paper_summary: The paper introduces R4D-planes, a new representation for dynamic scenes based on tensor decomposition aimed at improving novel view synthesis and self-supervised decoupling of monocular videos. The proposed method addresses the limitations of previous NeRF-based methods and explicit representations by introducing a remapping technique that fuses space-time information and compensates for the shortcomings of the plane structure. Additionally, a new decoupling structure is implemented to efficiently separate dynamic and static scenes. Experimental results indicate that R4D-planes achieve better rendering quality and training efficiency in both tasks compared to existing methods.
strength_weakness: strength_weakness: 1. The paper tackles important challenges in computer vision and computer graphics, specifically in the areas of view synthesis and dynamic scene decoupling. 
2. Experiments on two datasets demonstrate the superiority of the proposed method.
limitations: limitations: 1. The paper lacks experiments on the Plenoptic Video dataset, which is a significant omission given the relevance of this dataset to the tasks at hand. Including results from this dataset would provide a more comprehensive evaluation of the proposed method's performance.
2. There is a lack of detailed ablation studies to demonstrate the effectiveness of the remapping technique. The paper should also provide a thorough analysis of why it improves performance. What limitations of the previous methods that can be addressed by the remapping technique are not clear.
3. From Figure 4, it is difficult to judge which one is better, HyperNeRF or the method proposed in this paper. The authors are encouraged to zoom in on the details so that readers can quickly focus on the distinguished areas.
4. The authors claim that deformable 3DGS and 4DGS cannot model rapid motions in the scene, while the proposed method can accurately reconstruct them. However, one cannot make such a conclusion from the Figure 4 in the sumplementary materials. Besides, if a video demo can be provided in the sumplement, it would be easier for readers to observe the superiority of the proposed method.  
5. In Table 2, the PSNR of V4D should not be boldfaced.
paper_summary: paper_summary: This article introduces a novel approach for video novel view synthesis through the decoupling of static and dynamic objects using an Nd-Plan framework. The central innovation lies in the remapping function of the temporal dimension, which employs an activation function to distinguish between static and dynamic objects.
strength_weakness: strength_weakness: The proposed remapping extension improve the Nd-Plan NeRF framework in a straightforward and logical way by reducing the complexity of space-time representation with the mapping function without adding extra "hack". The article is clearly written and very easy to understand thanks to the relevance of the idea. The improvement in the evaluation is good enough to demonstrate the impact of the remapping function when using Nd-Plan framwork.
limitations: limitations: While the limitations discussed in the article are valid, there are additional concerns. This method may perform poorly in scenarios where objects move slowly in the whole scene or have low velocity, such as a tree swaying in the wind. the remapping function might have drawbacks in image rendering when compared to the traditional 4D K-Plan approach. These cases should be addressed by the authors in someway (adding an extra evaluation or a short paragraph on it). Additionally, the XZ plane splitting in Figure 1 is unclear for me.
paper_summary: paper_summary: This paper proposes a novel dynamic scene representation method based on tensor decomposition. By fusing spatio-temporal information and remapping to a new index, we compensate for the lack of planar structure. Experimental results show that R4D-planes outperform existing methods in terms of rendering quality and training efficiency for monocular scenes, as well as in dynamic and static decoupling tasks.
strength_weakness: strength_weakness: 1. **High quality rendering results**. 3D scene rendering has always been a very important research field. The method proposed by R4D-planes improves the rendering quality and training efficiency, and has strong generalization. It can be applied to many fields such as augmented reality/virtual reality (AR/VR), movies and games, providing new technical means for 3D scene reconstruction and rendering in these fields.

2. **New processing method and loss function**. The authors propose a dynamic monocular reconstruction method based on remapping and a dynamic and static decoupling method based on tensor decomposition, and give the derivation formula and loss function in detail, which enhances the generalization of the paper.

3. **Extensive experiments.** Extensive experiments shows its effectiveness.
limitations: limitations: 1. **Adaptability to complex scenarios:** Although this paper demonstrates excellent performance on specific datasets, the adaptability and robustness of R4D-planes in more complex and variable real-world scenarios (e.g., the ability to process complex motion patterns of dynamic objects) are not discussed in detail in this paper, and the adaptability and robustness of R4D-Planes need to be more extensively verified on different types of datasets.

2. **Limited ablation studies**. On the decoupling task, the influence of several proposed methods (such as different density initialization strategies, coarse-to-fine training schemes, decoder selection, and window position encoding) on the results is not explained in detail, which is expected to have a better qualitative representation.

3. **Complex methods design and background**. The proposed remapping and plane decoupling methods have a certain complexity in theory and implementation, which may increase the threshold of understanding and implementation. It is hoped that the introduction of the corresponding background knowledge, such as the definition of parameters and calculation methods, can be added in Section III.
paper_summary: paper_summary: This article proposes a novel method for synthesizing dynamic scene views by decoupling dynamic static backgrounds and dynamic remapping. And based on tensor decomposition, it is beneficial to establish a compact and continuous time representation.  The experimental results also verified the dynamic and static decomposition ability of the model, and extensive experiments have proven the effectiveness of the method.
strength_weakness: strength_weakness: (1) The logic of the method seems to be correct, and the advantage of remapping has also been demonstrated in the ablation experiment.

(2) The explanation of the method is clear, and the relevant work citations are sufficient for me.

(3) The extensive efforts made in the experimental section are commendable. This effectively demonstrates the advantages of the proposed method.
limitations: limitations: (1) Although a large number of visual comparison results have been provided, dynamic synthesized videos have not been provided, whether from fixed or free viewpoints. Unable to further evaluate the quality of video synthesis.

(2)How to understand Line 331's "a mechanism for sharing information across frames" ? The model seems to have no cross-frame information exchange.

(3) The model seems to be an improvement on HexPlane, but the difference between the two is not reflected in the visualization results.

(4) Lack of comparison between training time and storage costs. Although decoupling training time is shown in Table 3, it does not seem to be the training time for the entire model. Qualitative analysis of the latest methods such as 4DGS, HexPlane, KPlane, etc. should be added, and these methods have already been publicly available in code.