Original: Streaming Radiance Fields for 3D Video Synthesis
Revised: Streaming Radiance Fields for 3D Video Synthesis

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We present an explicit-grid based method for efficiently reconstructing streamingradiance fields for novel view synthesis of real world dynamic scenes.
Revised: We present an explicit-grid based method for efficiently reconstructing streamingradiance fields for novel view synthesis of real world dynamic scenes.

Original: Instead oftraining a single model that combines all the frames, we formulate the dynamicmodeling problem with an incremental learning paradigm in which per-framemodel difference is trained to complement the adaption of a base model on thecurrent frame.
Revised: Instead oftraining a single model that combines all the frames, we formulate the dynamicmodeling problem with an incremental learning paradigm in which per-framemodel difference is trained to complement the adaption of a base model on thecurrent frame.

Original: By exploiting the simple yet effective strategy of tuning with narrowbands, the proposed method achieves a feasible framework for handling on-thefly video sequences with high training efficiency.
Revised: By exploiting the simple yet effective strategy of tuning with narrowbands, the proposed method achieves a feasible framework for handling on-thefly video sequences with high training efficiency.

Original: The storage overhead inducedby using explicit grid can be significantly reduced by using model differencebased compression.
Revised: The storage overhead inducedby using explicit grid can be significantly reduced by using model differencebased compression.

Original: We also introduce a curriculum learning strategy to furtheraccelerate model optimization for each frame.
Revised: We also introduce a curriculum learning strategy to furtheraccelerate model optimization for each frame.

Original: Experiments on challenging videosequences demonstrate that our approach is capable of achieving a training speedof 10 seconds per-frame with competitive rendering quality, which attains 1500 ×speedup over the state-of-the-art implicit methods.
Revised: Experiments on challenging videosequences demonstrate that our approach is capable of achieving a training speedof 10 seconds per-frame with competitive rendering quality, which attains 1500 ×speedup over the state-of-the-art implicit methods.

Original:  Introduction
Revised:  Introduction

Original: 3D video synthesis aims to realize free-viewpoint photo-realistic rendering for dynamic scenes,which are typically recorded via a set of cameras (with known poses) from multiple views and hasattracted much research effort due to its potential value in a wide range of applications in VR/AR.
Revised: 3D video synthesis aims to realize free-viewpoint photo-realistic rendering for dynamic scenes,which are typically recorded via a set of cameras (with known poses) from multiple views and hasattracted much research effort due to its potential value in a wide range of applications in VR/AR.

Original: Traditional techniques like mesh [1, 2], multi-sphere image [3] and depth [4] estimation are usuallyused for modeling and representing dynamic scenes.
Revised: Traditional techniques like mesh [1, 2], multi-sphere image [3] and depth [4] estimation are usuallyused for modeling and representing dynamic scenes.

Original: However, the arbitrary geometry and complexappearance exhibited in real-world scenarios pose challenges for using a general methodology topursue high-performance modeling.
Revised: However, the arbitrary geometry and complexappearance exhibited in real-world scenarios pose challenges for using a general methodology topursue high-performance modeling.

Original: Neural radiance fields (NeRF) [5, 6] have recently emerged as a new methodology for effectivelyreconstructing and rendering static scenes via neural networks.
Revised: Neural radiance fields (NeRF) [5, 6] have recently emerged as a new methodology for effectivelyreconstructing and rendering static scenes via neural networks.

Original: By learning a continuous mappingbetween 3D points (given view directions) and its corresponding RGB radiance color and opacityequipped with volumetric rendering techniques [7], these methods have shown compelling ability forrendering high-fidelity images while encountering the bottleneck of costly training and inference asrelying on a tremendous amount of computations through networks.
Revised: By learning a continuous mappingbetween 3D points (given view directions) and its corresponding RGB radiance color and opacityequipped with volumetric rendering techniques [7], these methods have shown compelling ability forrendering high-fidelity images while encountering the bottleneck of costly training and inference asrelying on a tremendous amount of computations through networks.

Original: When extending the paradigm toa dynamic scene, time steps are embedded as additional input [8] to achieve time-dependent rendering.
Revised: When extending the paradigm toa dynamic scene, time steps are embedded as additional input [8] to achieve time-dependent rendering.

Original: However, the training overhead linearly increases with time, leading to enormous training costs whendealing with long sequences (e.g., costing about 56 GPU days for training 300 frames).
Revised: However, the training overhead linearly increases with time, leading to enormous training costs whendealing with long sequences (e.g., costing about 56 GPU days for training 300 frames).

Original: The implicitformulation essentially suiting offline processing may not be easily generalized to handling onlinevideo sequences.
Revised: The implicitformulation essentially suiting offline processing may not be easily generalized to handling onlinevideo sequences.

Original: In this work, we propose a novel method for reconstructing streaming radiance fields for novel view synthesis of 3D videos.
Revised: In this work, we propose a novel method for reconstructing streaming radiance fields for novel view synthesis of 3D videos.

Original: We formalize the modeling of a dynamic scene in an incremental learning
Revised: We formalize the modeling of a dynamic scene in an incremental learning

Original: These novel view result are rendered at interactive speed ( ∼10 FPS) by StreamRF.
Revised: These novel view result are rendered at interactive speed ( ∼

Original:  framework, enabling it to tackle on-the-fly video sequences in a streaming manner.
Revised: FPS) by framework, enabling it to tackle on-the-fly video sequences in a streaming manner.

Original: Inspired by therecent success achieved by Plenoxel [9] on training and inference efficiency, our method is built onlearning of explicit grid representations.
Revised: Inspired by therecent success achieved by Plenoxel [9] on training and inference efficiency, our method is built onlearning of explicit grid representations.

Original: Specifically, the overall training objective is decomposedto complete training of a basic voxel grid model at the first frame with an incrementally tuning onmodel difference in subsequent frames.
Revised: Specifically, the overall training objective is decomposedto complete training of a basic voxel grid model at the first frame with an incrementally tuning onmodel difference in subsequent frames.

Original: Only the model differences are expected to be trained andstored for each frame.
Revised: Only the model differences are expected to be trained andstored for each frame.

Original: A complete grid model given a time step i is simply attained by adding themodel difference to the previous model at i − 1 (or accumulating all the model differences before iwith the basic grid model).
Revised: A complete grid model given a time step i is simply attained by adding themodel difference to the previous model at i − 1 (or accumulating all the model differences before iwith the basic grid model).

Original: By exploiting the prior of exhibiting local continuity between adjacentframes, we present a narrow-band tuning strategy for capturing and training the model differenceefficiently, associated with a compression strategy for significantly reducing memory cost whichmay plague most explicit-grid methods.
Revised: By exploiting the prior of exhibiting local continuity between adjacentframes, we present a narrow-band tuning strategy for capturing and training the model differenceefficiently, associated with a compression strategy for significantly reducing memory cost whichmay plague most explicit-grid methods.

Original: To further improve the training efficiency, we introducea curriculum learning paradigm for learning pilot models to guide the optimization of full-scalemodels.
Revised: To further improve the training efficiency, we introducea curriculum learning paradigm for learning pilot models to guide the optimization of full-scalemodels.

Original: The overall framework is simple yet efficient.
Revised: The overall framework is simple yet efficient.

Original: Quantitative and qualitative experimentalresults validate the effectiveness of our approach.
Revised: Quantitative and qualitative experimentalresults validate the effectiveness of our approach.

Original: The approach can achieve the training speed of 10seconds and inference time of 100 ms per frame for rendering a 1k resolution image with competitivequality, achieving 1,500 times speedup over the state-of-the-art dynamic neural radiance field methods(costing 56 GPU-days for training a sequence with 300 frames).
Revised: The approach can achieve the training speed of 10seconds and inference time of 100 ms per frame for rendering a 1k resolution image with competitivequality, achieving 1,500 times speedup over the state-of-the-art dynamic neural radiance field methods(costing 56 GPU-days for training a sequence with 300 frames).

Original: It is also able to obtain 135 timesacceleration with a much lower storage cost compared to per-frame explicit-grid benchmarks.
Revised: It is also able to obtain 135 timesacceleration with a much lower storage cost compared to per-frame explicit-grid benchmarks.

Original: Theempirical analysis also demonstrates that using such a proper incremental training mechanism wouldnot degrade the performance in the scenarios of dealing with relatively long sequences.
Revised: Theempirical analysis also demonstrates that using such a proper incremental training mechanism wouldnot degrade the performance in the scenarios of dealing with relatively long sequences.

Original:  Problem Formulation
Revised:  Problem Formulation

Original: Neural Radiance fields (NeRF) [5] model a static scene implicitly by learning a continuous functionmapping from a 3D spatial position x ∈ R 3 specified in a viewing direction d ∈ R 3 (represented as
Revised: Neural Radiance fields (NeRF) [5] model a static scene implicitly by learning a continuous functionmapping from a 3D spatial position x ∈ R 3 specified in a viewing direction d ∈ R 3 (represented as

Original: Cartesian unit vectors) to RGB radiance color c ∈ R 3 and opacity σ ∈ R .
Revised: Cartesian unit vectors) to RGB radiance color c ∈ R 3 and opacity σ ∈ R .

Original: The mapping functionis typically implemented by multilayer perceptron (MLP) networks, f m : ( x , d ) (cid:55)→ ( c , σ ) , trainedwith a set of images with known camera pose.
Revised: The mapping functionis typically implemented by multilayer perceptron (MLP) networks, f m : ( x , d ) (cid:55)→ ( c , σ ) , trainedwith a set of images with known camera pose.

Original: Given a ray r casting on the image plane through apixel, by using a classical volume rendering technique [10], the expected color ˆ C ( r ) of the pixel isrendered by integrating the opacity and the color of sampling points along the ray:
Revised: Given a ray r casting on the image plane through apixel, by using a classical volume rendering technique [10], the expected color ˆ C ( r ) of the pixel isrendered by integrating the opacity and the color of sampling points along the ray:

Original: [Equation]
Revised: [Equation]

Original: where δ i denotes the distance to adjacent points.
Revised: where δ i denotes the distance to adjacent points.

Original: These methods are subjected to slow training andrendering speed, since ray casting involved in both training and rendering stage requires hundreds of forward propagation (and backward propagation in training) through the MLPs for the samplingpoints on each ray.
Revised: These methods are subjected to slow training andrendering speed, since ray casting involved in both training and rendering stage requires hundredsof forward propagation (and backward propagation in training) through the MLPs for the sampling points on each ray.

Original: For example, JaxNeRF[11], though optimized through parallelization, needs over20 seconds to render with at least one day of training.
Revised: For example, JaxNeRF[11], though optimized through parallelization, needs over20 seconds to render with at least one day of training.

Original: When extending the mechanism to model a dynamic scene, one straightforward solution is to embedthe time variable t into the representation, i.e., f t : ( x , d , t ) (cid:55)→ ( c , σ ) , where the training timewould linearly increase with respect to t .
Revised: When extending the mechanism to model a dynamic scene, one straightforward solution is to embedthe time variable t into the representation, i.e., f t : ( x , d , t ) (cid:55)→ ( c , σ ) , where the training timewould linearly increase with respect to t .

Original: Training a video with NeRF frameworks may cost over2,000 GPU-days.
Revised: Training a video with NeRF frameworks may cost over2,000 GPU-days.

Original: Although some variants, e.g., N3DV [8], improve training efficiency by virtueof sophistical designs, it still needs about 56 GPU-days in training to obtain a model with reliablerendering quality.
Revised: Although some variants, e.g., N3DV [8], improve training efficiency by virtueof sophistical designs, it still needs about 56 GPU-days in training to obtain a model with reliablerendering quality.

Original: More importantly, these implicit formulations are naturally suitable for offlineprocessing which feeds all the (key)frames in advance for training, and may be not easily generalizedto online training scenarios.
Revised: More importantly, these implicit formulations are naturally suitable for offlineprocessing which feeds all the (key)frames in advance for training, and may be not easily generalizedto online training scenarios.

Original: On the other hand, recent progress, representative by Plenoxel [9], has demonstrated compellingimprovement on training and render efficiency for modelling a static scene.
Revised: On the other hand, recent progress, representative by Plenoxel [9], has demonstrated compellingimprovement on training and render efficiency for modelling a static scene.

Original: Specially, a sparse voxelgrid is leveraged to replace the implicit representations in NeRF so that the scene is representedexplicitly.
Revised: Specially, a sparse voxelgrid is leveraged to replace the implicit representations in NeRF so that the scene is representedexplicitly.

Original: Each voxel in the grid consists of the scalar opacity σ and the coefficient vector k ofspherical harmonics (SH) for describing color information, which are directly optimized withoutneural networks.
Revised: Each voxel in the grid consists of the scalar opacity σ and the coefficient vector k ofspherical harmonics (SH) for describing color information, which are directly optimized withoutneural networks.

Original: The color at the voxel v is obtained by,
Revised: The color at the voxel v is obtained by,

Original: [Equation]
Revised: [Equation]

Original: where ξ is the sigmoid function and { Y ml } denotes the set of SH basis.
Revised: where ξ is the sigmoid function and { Y ml } denotes the set of SH basis.

Original: To render the expected colorof the pixel through a ray, the opacity and the color of a sampling point x is calculated by performingtrilinear interpolation among the voxels within its neighborhood N ( x ) ,
Revised: To render the expected colorof the pixel through a ray, the opacity and the color of a sampling point x is calculated by performingtrilinear interpolation among the voxels within its neighborhood N ( x ) ,

Original: [Equation]
Revised: [Equation]

Revised: v c ) .

Original: Then the pixel color is produced by integrating the sampling points along the ray based on Eq.
Revised: Then the pixel color is produced by integrating the sampling points along the ray based on Eq.

Original: Thevoxels with zero opacy (or truncated by a minimal threshold) are pruned after sparsification, and anoccupancy mask M with binary values is used for indicating if the space is occupied.
Revised: Thevoxels with zero opacy (or truncated by a minimal threshold) are pruned after sparsification, and anoccupancy mask M with binary values is used for indicating if the space is occupied.

Original: More detailscan refer to the paper of Plenoxels [9].
Revised: More detailscan refer to the paper of Plenoxels [9].

Original: In theory, the mechanism could significantly reduce the overallcomputational complexity as it omits the requirement of numerous MLP-based costly computationduring training and inference.
Revised: In theory, the mechanism could significantly reduce the overallcomputational complexity as it omits the requirement of numerous MLP-based costly computationduring training and inference.

Original: It is able to train a static scene with good rendering quality within 15minutes on single GPU and render a 1 k resolution image in 100 ms.
Revised: It is able to train a static scene with good rendering quality within 15minutes on single GPU and render a 1 k resolution image in 100 ms.

Original: These efficiency gains pose thechance to develop a highly efficient framework for modeling and rendering a dynamic scene.
Revised: These efficiency gains pose thechance to develop a highly efficient framework for modeling and rendering a dynamic scene.

Original:  Streaming Radiance Fields for Dynamic Scenes
Revised:  Streaming Radiance Fields for Dynamic Scenes

Original: To realize sparse grids for the modeling of a dynamic scene, one intuitive way is to directly extend 3Dspatial dimensions (denoted by H × W × D ) with the 4-th dimension indicating time t for frames.
Revised: To realize sparse grids for the modeling of a dynamic scene, one intuitive way is to directly extend 3Dspatial dimensions (denoted by H × W × D ) with the 4-th dimension indicating time t for frames.

Original: However, using explicit grids for a static scene may have faced the challenge on storage, althoughprogressive training and sparsifying (i.e, pruning off empty voxels) [9] are used to alleviate the issueof costly storage.
Revised: However, using explicit grids for a static scene may have faced the challenge on storage, althoughprogressive training and sparsifying (i.e, pruning off empty voxels) [9] are used to alleviate the issueof costly storage.

Original: Simply expanding the grid into a space-time manner would make the storageoverhead increase linearly with t , which is infeasible for handling videos with long sequences.
Revised: Simply expanding the grid into a space-time manner would make the storageoverhead increase linearly with t , which is infeasible for handling videos with long sequences.

Original: Onthe other hand, per-frame training individually without considering temporal correlation may befeasible for achieving a tradeoff between storage and efficiency, while it is still far from the objectiveof pursing a high-performing 3D video synthesis framework.
Revised: Onthe other hand, per-frame training individually without considering temporal correlation may befeasible for achieving a tradeoff between storage and efficiency, while it is still far from the objectiveof pursing a high-performing 3D video synthesis framework.

Original: To this end, we propose a novel method, named StreamRF, that realizes a streaming radiance field fordynamic scene rendering.
Revised: To this end, we propose a novel method, named StreamRF, that realizes a streaming radiance field fordynamic scene rendering.

Original: Our approach is built on the benefits of utilizing an explicit representationvia sparse voxel grids.
Revised: Our approach is built on the benefits of utilizing an explicit representationvia sparse voxel grids.

Original: Particularly, by leveraging some intrinsic priors exhibiting in videos, wedevelop an incremental training framework which enables a faster training convergence and lowerstorage over per-frame grid modelling.
Revised: Particularly, by leveraging some intrinsic priors exhibiting in videos, wedevelop an incremental training framework which enables a faster training convergence and lowerstorage over per-frame grid modelling.

Original: Videos of forward-facing scenarios are typically composed of time-invariant components (e.g., staticbackground) and time-variant components (e.g., moving objects) with small variation exhibiting local continuity between adjacent frames.
Revised: Videos of forward-facing scenarios are typically composed of time-invariant components (e.g., staticbackground) and time-variant components (e.g., moving objects) with small variation exhibiting local continuity between adjacent frames.

Original: By exploiting the fact, we decompose the sequentialmodeling problem into two sub-problems.
Revised: By exploiting the fact, we decompose the sequentialmodeling problem into two sub-problems.

Original: We first fully train a base grid model given the firstframe, denoted as V 

Revised: We first fully train a base grid model given the firstframe, denoted as V

Original: = { V 0 c , V 0 σ } which stores the corresponding color features (the coefficients ofspherical harmonics) and opacity values.
Revised: = { V 0 c , V 0 σ } which stores the corresponding color features (the coefficients ofspherical harmonics) and opacity values.

Original: Then the model is adapted to succeeding frames throughthe use of an effective tuning strategy.
Revised: Then the model is adapted to succeeding frames throughthe use of an effective tuning strategy.

Original: Per-frame tuning is expected to perform only on the changesbetween adjacent frames.
Revised: Per-frame tuning is expected to perform only on the changesbetween adjacent frames.

Original: The grid model in the current frame is produced by adding the modeldifference, which is optimized and stored during training phase, to the grid model in the previousframe.
Revised: The grid model in the current frame is produced by adding the modeldifference, which is optimized and stored during training phase, to the grid model in the previousframe.

Original: Formally, assume the model difference between the i − 1 -th and i -th frames is denoted asδ i =
Revised: Formally, assume the model difference between the i − 1 -th and i -th frames is denoted asδ i =

Original: ∆ ( V i , V i − 1 ) .
Revised: ∆ ( V i , V i − 1 ) .

Original: In the inference phase, the grid model at the time step i can be obtained by,
Revised: In the inference phase, the grid model at the time step i can be obtained by,

Original: [Equation]
Revised: [Equation]

Original: Such a incremental formulation indeed presents an online training paradigm, enabling the proposedframework to handle on-the-fly video sequences.
Revised: Such a incremental formulation indeed presents an online training paradigm, enabling the proposedframework to handle on-the-fly video sequences.

Original: 3.1 Tuning with Narrow Bands
Revised: 3.1 Tuning with Narrow Bands

Original: A problem arises: how to effectively capture the model difference between adjacent frames i − 1and i .
Revised: A problem arises: how to effectively capture the model difference between adjacent frames i − 1and i .

Original: Given the sparse grid model V i − 1 , any object motion at the time step i may incur changes onthe opacity or colors of a part of voxels.
Revised: Given the sparse grid model V i − 1 , any object motion at the time step i may incur changes onthe opacity or colors of a part of voxels.

Original: It is hence likely to happen that informative points occurin the originally empty space as well as some voxels that are originally occupied becomes empty.
Revised: It is hence likely to happen that informative points occurin the originally empty space as well as some voxels that are originally occupied becomes empty.

Original: Directly tuning the sparse grid can only change or one-way prune the non-empty voxels, intractableto handling long sequences.
Revised: Directly tuning the sparse grid can only change or one-way prune the non-empty voxels, intractableto handling long sequences.

Original: To address the issue, we propose a tuning strategy based on narrow bands.
Revised: To address the issue, we propose a tuning strategy based on narrowbands.

Original: We first retrieve andactivate the narrow band for frame i , i.e., the region of interest in the grid
Revised: We first retrieve and activate the narrow band for frame i , i.e., the region of interest in the grid

Original: R i , based on the occupancymask M i − 1 with binary values which indicates non-empty voxels at the frame i − 1 ,
Revised: R i , based on the occupancy mask M i − 1 with binary values which indicates non-empty voxels at theframe i − 1 ,

Original: [Equation]
Revised: [Equation]

Original: where F di and F er correspond to dilation and erosion operations, and ρ d and ρ e denote the corresponding radius parameters.
Revised: where F di and F er correspond to dilation and erosion operations, and ρ d and ρ e denote the corresponding radius parameters.

Original: ⊕ denotes the element-wise xor operation on the two masks to extractthe additional region of the dilated area compared to the eroded one.
Revised: ⊕ denotes the element-wise xor operation on the two masks to extractthe additional region of the dilated area compared to the eroded one.

Original: The narrow band actually represents a thin area around the surface of scenes (or objects in a compositescene).
Revised: The narrow band actuallyrepresents a thin area around the surface of scenes (or objects in a composite scene).

Original: We use it for two purposes: • For every empty voxels fall into the band, we restore them to the grid and initialize the opacity and color features with zero values.
Revised: We use it fortwo purposes: (1) For every empty voxels fall into the band, we restore them to the grid and initializethe opacity and color features with zero values.

Original: • Only the voxels within the band are tuned and the rest ones (with a large number) are notinvolved during the training stage for the frame i .
Revised: (2) Only the voxels within the band are tuned and therest ones (with a large number) are not involved during the training stage for the frame i .

Original: We also prune empty voxels to get a compact grid
Revised: We alsoprune empty voxels to get a compact grid

Original: V i with an occupancy mask M i after tuning in theframe i .
Revised: V i with an occupancy mask M i after tuning in the frame i .

Original: The key idea of the narrow band-based tuning is built upon leveraging the intrinsic priors that motionsinduced by moving objects typically show small changes and local continuity within neighboringareas between adjacent frames.
Revised: The key idea of the narrow band-based tuning is built upon leveraging the intrinsic priors that motionsinduced by moving objects typically show small changes and local continuity within neighboringareas between adjacent frames.

Original: These assumptions have been successfully exploited in a widerange of video processing [12] and understanding tasks [13, 14], e.g, traditional mean-shift methodfor tracking [15].
Revised: These assumptions have been successfully exploited in a wide range of video processing [12] and understanding tasks [13, 14], e.g, traditional mean-shift methodfor tracking [15].

Original: We demonstrate that the strategy can effectively benefit to training convergence,capable of achieving the speed of per-frame ∼ 10 s training for achieving a competitive quality (shownin section 5.4).
Revised: We demonstrate that the strategy can effectively benefit to training convergence,capable of achieving the speed of per-frame ∼ 10 s training for achieving a competitive quality (shownin section 5.4).

Original: Compared to the state-of-the-art implicit method [8] with the training speed of 4.

Revised: Compared to the state-of-the-art implicit method [8] with the training speed of 4.

Original: GPU-hours for each frame in average (costing about 56 GPU-days for training a sequence of 300frames), our method can achieve over 1,500 times speedup on trainingand over 200 times speedupon rendering efficiency (25 s VS 121 ms).
Revised: GPU-hours for each frame in average (costing about 56 GPU-days for training a sequence of frames), our method can achieve over 1,500 times speedup on training over 200 times speedup on rendering efficiency (25 s VS 121 ms).

Original: 3.2 Difference-based Compression
Revised: 3.2 Difference-based Compression

Original:  Using explicit grid modelling yet poses the issue of large storage cost.
Revised: [Figure] Using explicit grid modelling yet poses the issue of large storage cost.

Original: Taking a grid size ( 1408 ×1156 × 128 in the experimental setting) for example, the overall storage costs about 5.4 MB aftersparsity for one-frame model, as a conventional sparse grid.
Revised: Taking a grid size ( 1408 ×1156 × 128 in the experimental setting) for example, the overall storage costs about 1015 MB aftersparsity for one-frame model, as a conventional sparse grid.

Original: By virtue of narrow band tuning, we onlyneed to store the model difference δ i with a significantly smaller size over the complete grid model.
Revised: By virtue of narrow band tuning, we onlyneed to store the model difference δ i with a significantly smaller size over the complete grid model.

Original: The change of storage cost only happen in the following two scenarios, i.e., when adding voxels intothe grid (i.e., appearing in the empty space) and pruning voxels from the grid (i.e., the previouslyoccupied space become empty).
Revised: The change of storage cost only happen in the following two scenarios, i.e., when adding voxels intothe grid (i.e., appearing in the empty space) and pruning voxels from the grid (i.e., the previouslyoccupied space become empty).

Original: In this regard, we introduce a difference-based compression strategyto further reduce the storage overhead.
Revised: In this regard, we introduce a difference-based compression strategyto further reduce the storage overhead.

Original: Particularly, we can efficiently find out the change areas byusing the occupancy masks M i − 1 and M i between the adjacent frames i − 1 and i ,
Revised: Particularly, we can efficiently find out the change areas byusing the occupancy masks M i − 1 and M i between the adjacent frames i − 1 and i ,

Original: [Equation]
Revised: [Equation]

Original:  where M i a , M i e and M i r representatively denote the masks of adding voxels, erasing voxels andremaining ones while the features at the voxels may change.
Revised: 6) where M i a , M i e and M i r respectively denote the masks of adding voxels, erasing voxels and remainingones while the features at the voxels may change.

Original: γ ( · ) defines the function of getting the L1 norm along the axis of color features (SH coefficients) on the model difference, and ξ denotesa threshold.
Revised: γ ( · ) defines the function of getting the L1 normalong the axis of color features (SH coefficients) on the model difference, and ξ denotes a threshold.

Original: M ir actually involves the voxels with relatively large modification.
Revised: M ir actually involves the voxels with relatively large modification.

Original: Instead of storingmodel difference δ i directly, we store the three masks, the voxel representations (SH coefficients and opacity) belonging to the adding area M ia and the representation differences for the voxels within theremaining area M ir .
Revised: Instead of storing model differenceδ i directly, we store the three masks, the voxel representations (SH coefficients and opacity) belongingto the adding area M ia and the representation differences for the voxels within the remaining area M ir .

Original: It can achieve a remarkable reduction rate (187 times) on storage that reduces1015 MB to 5.4{+5.
+} MB for each frame in average.
Revised: It can achieve a remarkable reduction rate (187 times) on storage that reduces 1015 MB to {+5.
+} MB for each frame in average.

Original:  Efficient Training with Pilot Model Guidance
Revised:  Efficient Training with Pilot Model Guidance

Original: To further improve the training efficiency of StreamRF, we exploit a curriculum learning paradigmthat trains from an easier problem.
Revised: To further improve the training efficiency of StreamRF, we exploit a curriculum learning paradigmthat trains from an easier problem.

Original: Formally, given the grid model V i − 1 , we adopt down-samplingoperations to get a pilot model of a smaller size with the scale S ,(cid:98)
Revised: Formally, given the grid model V i − 1 , we adopt down-samplingoperations to get a pilot model of a smaller size with the scale S ,

Original: [Equation]
Revised: [Equation]

Original: When setting the S to 1 / 2 , the capacity of pilot model is roughly 1 / 8 of the full grid.
Revised: When setting the S to 1 / 2 , the capacity of pilot model is roughly 1 / 8 of the full grid.

Original: Training on asmaller model would reduce the computational complexity at every iteration (with the same batchsize) and also ease optimization.
Revised: Training on asmaller model would reduce the computational complexity at every iteration (with the same batchsize) and also ease optimization.

Original: The model difference on the pilot models (cid:98) δ i =
Revised: The model difference on the pilot models (cid:98) δ i =

Original: ∆ ( (cid:98) V i , (cid:98) V i − 1 ) areimplemented by using { (cid:99) M ia , (cid:99) M ie , (cid:99) M ir } as following Eq. 6.
Revised: ∆ ( (cid:98) V i , (cid:98) V i − 1 ) areimplemented by using { (cid:99) M ia , (cid:99) M ie , (cid:99) M ir } as following Eq. 6.

Original: Then we can get a guidance mask (cid:98) G i bymerging the three mask via elementwise "or" operator and upsampling to the full scale, 
Revised: Then we can get a guidance mask (cid:98) G i bymerging the three mask via elementwise "or" operator and upsampling to the full scale, [Equation]

Original: [Equation]
Revised: [Equation]

Original: We adopt the well-trained pilot model with the guidance mask  for fine-tuning the full-scale gridmodel.
Revised: The guidance mask is used for regularizing the gradient update when tuning the full-scale model.

Revised: The values of added and erased voxels are filled back to the full-scale grid.

Original: All the voxels outside the guidance mask G i are frozen without optimization when training atthe frame i .
Revised: All the voxels outside themask are frozen without optimization when training at the frame i .

Revised: The design of the strategy comes from the insight that training at a small size converges much fasterand the induced difference of pilot model indicates the voxels that need to be modified.

Revised: Using thisstrategy can prevent unnecessary modification in the full-scale model and avoid the loss of highfrequency details during downsample-upsample process.

Original: We observe that using this strategy also facilitate pursuing a higher fidelity model compared to directlytraining on the full-scale grid with the same training times.
Revised: We observe that using this strategy facilitatepursuing a higher fidelity model compared to directly training on the full-scale grid with the sametraining time.

Original: Specially, the qualitative results (in Fig. 4)reveals that the model via directly training on the full-scale grid may encounter flicker issue whenrendering along time steps to a sequence, i.e., sparkling noise may occur in the stable backgroundarea, when training a short period.
Revised: Specially, the qualitative results (in Fig. 4) reveals that the model via directly trainingon the full-scale grid may encounter flicker issue when rendering along time steps to a sequence, i.e.,sparkling noise may occur in the stable background area, when training a short period.

Original: The issue could be addressed by lengthening the training schedulewith a small learning rate.
Revised: The issuecould be addressed by lengthening the training schedule with a small learning rate.

Original: While training with the pilot model guidance can effectively prohibit theflicker artifacts from happening. Training first on a down-sampled pilot grid essentially introduceslocal consistency for neighboring voxels when upsampling to the full-scale grid, enabling a morestable optimization for the streaming paradigm.
Revised: While trainingwith the pilot model guidance can effectively prohibit flicker artifacts from happening, enabling amore efficient and stable optimization for the streaming paradigm.

Original:  Experiments
Revised:  Experiments

Original: 5.1 Datasets
Revised: 5.1 Datasets

Original: Meet room Dataset .
Revised: Meet room Dataset .

Original: We build a multi-view capture system using 13 Azure Kinect cameras.
Revised: We build a multi-view capture system using 13 Azure Kinect cameras.

Original: Twelvecameras were used for training, and onecamera was used for testing. We use an external pulse signalto synchronize all camera shutter.
Revised: Weuse twelve cameras for training and one for testing, with an external pulse signal to synchronize allcamera shutter.

Original: Though this system is capable of depth capture, we do not usedepth results in this research.
Revised: Though this system is capable of depth capture, we do not use depth results in thisresearch.

Original: The capture videos were recorded at a resolution of 1280 × 720 anda frame rate of 30{+
+} FPS.
Revised: The capture videos were recorded at a resolution of 1280 × 720 and a frame rate of {+
+} FPS.

Original: The total size of the capture system was 100 cm × 75 cm, making it muchsparser than some common datasets such as LLFF and N3DV.
Revised: The total size of the capture system was 100 cm × 75 cm, making it much sparser than somecommon datasets such as LLFF and N3DV.

Original: We will release this dataset for researchpurposes.
Revised: We will release this dataset for research purposes.

Original: Neural3D Video Dataset[8] contains 6 dynamic indoor scenes with varying illuminations, viewdependent effects, and highly volumetric effects.
Revised: Neural3D Video Dataset[8] contains 6 dynamic indoor scenes with varying illuminations, viewdependent effects, and highly volumetric effects.

Original: Videos were captured at a resolution of 2704 × 2078and a frame rate of 30 FPS.
Revised: Videos were captured at a resolution of 2704 × 2078and a frame rate of 30 FPS.

Original: Following the N3DV paper, we use one view for testing and 18 views fortraining.
Revised: Following the N3DV paper, we use one view for testing and 18 views fortraining.

Original: We choose the designated sequence and downsample it to 1352 × 1039 for a fair comparisonof the quantitative score.
Revised: We choose the designated sequence and downsample it to 1352 × 1039 for a fair comparisonof the quantitative score.

Original: 5.2 Implementation Details
Revised: 5.2 Implementation Details

Original: Initialization.
Revised: Initialization.

Original: We follow the standard training protocol in a forward-facing scene of Plenoxels[9].
Revised: We follow the standard training protocol in a forward-facing scene of Plenoxels[9].

Original: We start by 256 × 256 × 128 grid, upsample and sparsify for every 38.4K iterations until the model [Table] reaches the final resolution of 1408 × 1156 × 128 .
Revised: We start by 256 × 256 × 128 grid, upsample and sparsify for every 38.4K iterations until the modelreaches the final resolution of 1408 × 1156 × 128 .

Original: The total training takes 128K iterations with abatch size of 5,000 rays.
Revised: The total training takes 128K iterations with abatch size of 5,000 rays.

Original: We adopt the RMSProp [16] optimizer with a decay parameter of β = 0 .
Revised: We adopt the RMSProp [16] optimizer with a decay parameter of β = 0 .

Original: 95for voxel grid optimization.
Revised: 95for voxel grid optimization.

Original: We also adopt a higher TV loss to mitigate the foggy issue, with λ TVset to 5 × 10 − 4 for opacity and 5 × 10 − 3 for color features.
Revised: We also adopt a higher TV loss to mitigate the foggy issue, with λ TVset to 5 × 10 − 4 for opacity and 5 × 10 − 3 for color features.

Original: Per Frame Tuning.
Revised: Per Frame Tuning.

Original: The main model and the pilot model are trained in parallel.
Revised: The main model and the pilot model are trained in parallel.

Original: We train the mainmodel with 500 iterations and the pilot model with 1,000 iterations.
Revised: We train the mainmodel with 500 iterations and the pilot model with 1,000 iterations.

Original: We use RMSprop to optimizeboth models with a batch size of 20,000 rays.
Revised: We use RMSprop to optimizeboth models with a batch size of 20,000 rays.

Original: To promote the pilot model to have a better convergence,we enlarge λ TV 10 times when tuning the pilot model while keeping it untouched for the main model.
Revised: To promote the pilot model to have a better convergence,we enlarge λ TV 10 times when tuning the pilot model while keeping it untouched [Table]

Original: Diff Based Compression.
Revised: Diff Based Compression.

Original: We set the SH coefficient threshold ξ to 1/27 (27 is the channel number of
Revised: We set the SH coefficient threshold ξ to 1/27 (27 is the channel number of

Original: SH coefficients).
Revised: SH coefficients).

Original: A smaller choice would drastically increase saving size with only marginal gain in
Revised: A smaller choice would drastically increase saving size with only marginal gain in

Original: PSNR.
Revised: PSNR.

Original: We convert all saving SH coefficient and density value to float16 to further decrease savingsize.
Revised: We convert all saving SH coefficient and density value to float16 to further decrease savingsize.

Original: In actual implementation we could merge M ia and M ie to one single mask to save more space.
Revised: In actual implementation we could merge M ia and M ie to one single mask to save more space.

Original: Moreover, as the mask is mostly consecutive 0 and 1, both masks have a very high compression ratio(over 99%) even with standard compression library ( e.g. ZLIB[17]).
Revised: Moreover, as the mask is mostly consecutive 0 and 1, both masks have a very high compression ratio(over 99%) even with standard compression library ( e.g. ZLIB[17]).

Original: We pack all tensors together andcompress them with ZLIB to get the final on-disk per frame storage.
Revised: We pack all tensors together andcompress them with ZLIB to get the final on-disk per frame storage.

Revised: Pilot Model Guidance.

Revised: We first downsample the grid model and apply the standard training pipeline(narrow band + difference-based compression) for pilot model training.

Revised: The induced model differenceis used in two purposes: 1) the values of added and erased voxels are filled back to the full-scalemodel. 2) the guidance mask is used to regularize gradient update for tuning the full-scale model.

Original: 5.3 Results
Revised: 5.3 Results

Original: We first quantitatively compare with benchmark methods in Table 1.
Revised: We first quantitatively compare with benchmark methods in Table 1.

Original: We use peak signal-to-noiseratio (PSNR), per frame training time, and inference time as evaluation metrics.
Revised: We use peak signal-to-noiseratio (PSNR), per frame training time, and inference time as evaluation metrics.

Original: We follow the N3DVtesting protocol [8] and choose “flame_salmon" as the test sequence.
Revised: We follow the N3DVtesting protocol [8] and choose “flame_salmon" as the test sequence.

Original: We evaluate the four methodsevery ten frames to calculate the mean value of per-frame metrics.
Revised: We evaluate the four methodsevery ten frames to calculate the mean value of per-frame metrics.

Original: We refer to the values of PSNRand training time reported in the paper of N3DV for a fair comparison.
Revised: We refer to the values of PSNRand training time reported in the paper of N3DV for a fair comparison.

Original: We estimate the renderingtime of N3DV according to its network design.
Revised: We estimate the renderingtime of N3DV according to its network design.

Original: Because they use a larger MLP than the one used in
Revised: Because they use a larger MLP than the one used in

Original: NeRF [5], its inference time is supposed to be longer than NeRF.
Revised: NeRF [5], its inference time is supposed to be longer than NeRF.

Original: As shown in the table, the two implicit methods JaxNeRF and N3DV reveal high computation costthat requires over 8 GPU hours and 4 GPU hours for per-frame training (N3DV costs about 

Revised: As shown in the table, the two implicit methods JaxNeRF and N3DV reveal high computation costthat requires over 8 GPU hours and 4 GPU hours for per-frame training (N3DV costs about

Original: GPU days to train 300 frames), respectively.
Revised: GPU days to train 300 frames), respectively.

Original: As for testing speed, the LLFF achieve the fastestrecord.
Revised: As for testing speed, the LLFF achieve the fastestrecord.

Original: Because it only needs blending the multi-plane image to render a novel view.
Revised: Because it only needs blending the multi-plane image to render a novel view.

Original: However, itfails to produce a satisfactory result.
Revised: However, itfails to produce a satisfactory result.

Original: We achieve the optimization speed of costing 10 seconds only,outperforming all the baseline methods in terms of training efficiency.
Revised: We achieve the optimization speed of costing 10 seconds only,outperforming all the baseline methods in terms of training efficiency.

Original: Meanwhile our method canachieve a comparable PSNR to the previous SOTA.
Revised: Meanwhile our method canachieve a comparable PSNR to the previous SOTA.

Original: For the meet room dataset, we use the samesequence as in Fig 2 for testing and following the same test protocol as above.
Revised: For the meet room dataset, we use the samesequence as in Fig 2 for testing and following the same test protocol as above.

Original: Because our datasetcontains fast motion and camera spacing is larger than N3DV dataset, we could see a clear PSNRdrop on all four methods.
Revised: Because our datasetcontains fast motion and camera spacing is larger than N3DV dataset, we could see a clear PSNRdrop on all four methods.

Original: Nevertheless, our method still achieves a comparable performance withper frame training counterpart, around 0.3 drop in PSNR compared to JaxNeRF and Plenoxels,while obtaining the acceleration over 2 , 700 × and 80 × , respectively.
Revised: Nevertheless, our method still achieves a comparable performance withper frame training counterpart, around 0.3 drop in PSNR compared to JaxNeRF and Plenoxels,while obtaining the acceleration over 2 , 700 × and 80 × , respectively.

Original: More experimental results andanalysis can be referred to supplementary materials.
Revised: More experimental results andanalysis can be referred to supplementary materials.

Original: [Figure]

Original: 5.4 Ablation study
Revised: 5.4 Ablation study

Original: Narrow Band Finetune We compare the narrow band with the following two counterparts, bothquantitative and visually: (1) directly adopt the original sparse voxels grid, (2) switch the originalsparse voxels grid to its dense counterpart.
Revised: Narrow Band Finetune We compare the narrow band with the following two counterparts, bothquantitative and visually: (1) directly adopt the original sparse voxels grid, (2) switch the originalsparse voxels grid to its dense counterpart.

Original: It can be seen in Fig.2.
Revised: It can be seen in Fig.2.

Original: The sparse grid failed immediately(Fig.2 (a)), though the right person only move at a small distance.
Revised: The sparse grid failed immediately(Fig.2 (a)), though the right person only move at a small distance.

Original: This artifact is because the subjecthas moved out of the trainable area.
Revised: This artifact is because the subjecthas moved out of the trainable area.

Original: A dense voxel grid allows every voxel to be trainable.
Revised: A dense voxel grid allows every voxel to be trainable.

Original: It stillfails when the right person has a large movement(Fig.2 (b)).
Revised: It still [Figure] fails when the right person has a large movement(Fig.2 (b)).

Original: This artifact could be caused by therapidly increasing number of active voxels after switching from sparse to dense (5% vs. 100% of allvoxels).
Revised: This artifact could be caused by therapidly increasing number of active voxels after switching from sparse to dense (5% vs. 100% of allvoxels).

Original: Both PSNR curve and rendering result show our method achieves comparable PSNR with adense grid and can handle large movement in a dynamic scene.
Revised: Both PSNR curve and rendering result show our method achieves comparable PSNR with adense grid and can handle large movement in a dynamic scene.

Original: Moreover, in this ablation study, ourmethod trains around 40% faster and requires 63% less memory consumption than dense finetune.
Revised: Moreover, in this ablation study, ourmethod trains around 40% faster and requires 63% less memory consumption than dense finetune.

Original: Diff-based Compression We validate the performance and effectiveness of diff-based compressionin Fig.
Revised: Diff-based Compression We validate the performance and effectiveness of diff-based compressionin Fig.

Original: In this ablation, we first train our StreamRF on an entire sequence.
Revised: In this ablation, we first train our StreamRF on an entire sequence.

Original: The trained StreamRFis then saved with diff-based compression enabled and disabled.
Revised: The trained StreamRFis then saved with diff-based compression enabled and disabled.

Original: We then compare the PSNR andsaving size between the two test view sequences.
Revised: We then compare the PSNR andsaving size between the two test view sequences.

Original: We present the per frame PSNR to show whetherdiff-based compression impairs the performance in the long run.
Revised: We present the per frame PSNR to show whetherdiff-based compression impairs the performance in the long run.

Original: The quantitative result shows that our compression method has negligible impact after a very longpropagation.
Revised: The quantitative result shows that our compression method has negligible impact after a very longpropagation.

Original: Besides deducting overall storage size from 1015 GB to an average of 5.4 MB foreach frame, we can observe that performance of our method does not downgrade with time and theincremental storage for each frame is roughly stable along time steps.
Revised: Besides deducting overall storage size from 1015 MB to an average of 5.4 MB foreach frame, we can observe that performance of our method does not downgrade with time and theincremental storage for each frame is roughly stable along time steps.

Original: Pilot Model Guidance We validate our pilot model guidance visually in Fig.
Revised: Pilot Model Guidance We validate our pilot model guidance visually in Fig.

Original: For base setting(right), we train the pilot model with 1,000 iterations and the full model with 500 iterations.
Revised: For base setting(right), we train the pilot model with 1,000 iterations and the full model with 500 iterations.

Original: For theablation setting (left), we directly train the full model with 1,000 iterations to make per frame trainingtime equivalent in both settings.
Revised: For theablation setting (left), we directly train the full model with 1,000 iterations to make per frame trainingtime equivalent in both settings.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4, pilot model guidance help reduce various kindsof artifacts: (top) vanishing hands and blurry elbow.
Revised: 4, pilot model guidance help reduce various kindsof artifacts: (top) vanishing hands and blurry elbow.

Original: (bottom) sparkle noise in a white wall, the fuzzyarea between two people.
Revised: (bottom) sparkle noise in a white wall, the fuzzyarea between two people.

Original: These results verified our intention that we could use an easy-to-optimizedsmall model to provide a more optimal initialization to the full model.
Revised: These results verified our intention that we could use an easy-to-optimizedsmall model to provide a more optimal initialization to the full model.

Original:  Related Work
Revised:  Related Work

Original: Static Novel View Synthesis Novel view synthesis from a set of input images with known cameraposes has been studied intensively in the field of computer vision and computer graphics.
Revised: Static Novel View Synthesis Novel view synthesis from a set of input images with known cameraposes has been studied intensively in the field of computer vision and computer graphics.

Original: Earlierworks like Lumigraph [19, 20] and Light-Fied [21, 22] focus on interpolating the sampled light raysfor realistic rendering.
Revised: Earlierworks like Lumigraph [19, 20] and Light-Fied [21, 22] focus on interpolating the sampled light raysfor realistic rendering.

Original: It has been proved [23, 24] that the sampling density heavily depends on theunderlying geometry of the captured scene.
Revised: It has been proved [23, 24] that the sampling density heavily depends on theunderlying geometry of the captured scene.

Original: Thus, many works strive to recover an approximate [25,26] or precise geometry model [27] to facilitate novel view rendering.
Revised: Thus, many works strive to recover an approximate [25,26] or precise geometry model [27] to facilitate novel view rendering.

Original: Among those works, multiplane image (MPI)
Revised: Among those works, multiplane image (MPI)

Original: [18, 28–30] is a promising approximate geometry model that enables realisticrendering of generic scenes.
Revised: [18, 28–30] is a promising approximate geometry model that enables realisticrendering of generic scenes.

Original: More recently, NeRF [5] represents the radiance field with a multi-layer perceptron (MLP) networkand combines it with differentiable volume rendering for view interpolation.
Revised: More recently, NeRF [5] represents the radiance field with a multi-layer perceptron (MLP) networkand combines it with differentiable volume rendering for view interpolation.

Original: This approach achievedtremendous success and there are many variants to push its limits towards every aspect.
Revised: This approach achievedtremendous success and there are many variants to push its limits towards every aspect.

Original: Theworks [6, 31–33] focus on improving rendering quality, while the works [34–36] strive to acceleraterendering speed, and other works [9, 37–39] propose to replace implicit function by an explicit orhybird counterpart to boost training convergence.
Revised: Theworks [6, 31–33] focus on improving rendering quality, while the works [34–36] strive to acceleraterendering speed, and other works [9, 37–40] propose to replace implicit function by an explicit orhybird counterpart to boost training convergence.

Original: Dynamic Novel View Synthesis Novel view synthesis has also been heavily studied with dynamicscenes.
Revised: Dynamic Novel View Synthesis Novel view synthesis has also been heavily studied with dynamicscenes.

Original: Early effort of generating 3D video can trace back to the 90s [1, 40], which computes explicitdepth maps to facilitate novel view interpolation.
Revised: Early effort of generating 3D video can trace back to the 90s [1, 41], which computes explicitdepth maps to facilitate novel view interpolation.

Original: This approach has been combined with moresophisticated capturing hardware to generate streamable free-viewpoint video in [2].
Revised: This approach has been combined with moresophisticated capturing hardware to generate streamable free-viewpoint video in [2].

Original: Matusik and
Revised: Matusik and

Original: Pfister [41] further integrate camera arrays and 3D displays to build an end-to-end 3D TV system.
Revised: Pfister [42] further integrate camera arrays and 3D displays to build an end-to-end 3D TV system.

Original: More recent works learn a neural network to capturethe radiance field ofa dynamic scene.
Revised: More recent works learn a neural network to capture radiance fields of dynamic scenes.

Original: Some ofthem [42–44] focus on monocular video input for novel view synthesis at nearby viewpoints.
Revised: A familyof methods [43–49] realize view synthesis by training on monocular videos captured by a movingcamera, i.e., viewpoints change with respect to time steps.

Original: Li etal. [42] generalize the original NeRF model to  a neural scene flow field to deal with dynamic scenes.
Revised: The works in [46–48] decouple theoverall radiance field reconstruction to the learning of a static canonical template and a deformationfield warping to the template.

Original: Xian et al.
Revised: Park et al.

Original: [43] bring depth prior to better estimate the 4D radiance filed. Gao et al. [44] jointly trainan dynamic NeRF and static NeRF to better preserve details at both part.
Revised: [49] lifts canonical templates into hyper-spaces to induceflexibility on modelling variations in topology. Several methods model dynamics through the useof scene flows [43, 45] or depth prior [44], therefore they highly reply on the regularization ofextra supervision (e.g., optical flow, depth or foreground-background segmentation estimated bywell-trained models).

Original: In compraison, Broxtonet al.
Revised: Broxton et al.

Original: [3] and Attal et al.
Revised: [3] and Attal et al.

Original: [45] use multi-camera stereo rig and extendMPI (Multi Plane Image) to
Revised: [50] use multi-camera stereo rig and extend

Original: MSI(Multi Sphere Image) to support view interpolation with  larger field of view.
Revised: MPI to MSI to support view interpolation with a larger field of view.

Original: All these abovemethods only support small movement of camera viewpoints.

Original: Bansal et al.
Revised: Bansal et al.

Original: [46] use multiple mobile phones to capture a dynamic scene and compose the static anddynamic components with neural networks for 4D space-time visualization.
Revised: [51] use multiplemobile phones to capture a dynamic scene and compose the static and dynamic components withneural networks for 4D space-time visualization.

Original: The closest work to oursetting is Neural 3D Video (N3DV)[8], which  directly extends NeRF with an additional timecodeinput, and accelerates computation with a hybrid importance sampling and hierarchical training.
Revised: The closest work to our setting is Neural 3D Video (N3DV)[8], which captures dynamic scenes with synchronous video sequences from multiplecameras. It directly extends NeRF with an additional timecode input, and accelerates computationwith a hybrid importance sampling and hierarchical training.

Original: N3DV can produce strong visual results given enough optimization. However, N3DV is very timeconsuming, requiring over  5 GPU hours per frame in training, and 10+ secondsper frame in rendering,making it impractical for real-world applications.
Revised: N3DV can produce strong visual resultsgiven enough optimization, while it is very time consuming, requiring over per frame 5 GPU hours intraining and 10+ seconds in rendering, making it impractical for real-world applications.

Revised: On the other hand, some approaches adapt the advances in neural rendering for the modeling ofspecific objects, e.g., human bodies [52, 53] by integrating with domain-specific priors, e.g., skeletonor SMPL mesh [54].

Revised: In contrast, our method imposes no prior in objects and is capable of representingmore general scenes with complex deformation.

Original:  Conclusion
Revised:  Conclusion

Original: We propose a novel streaming radiance field method for effectively reconstructing and renderingdynamic scenes with explicit grid representations.
Revised: We propose a novel streaming radiance field method for effectively reconstructing and renderingdynamic scenes with explicit grid representations.

Original: The modelling of dynamic scenes is formalized asan incremental learning paradigm which allows the method for handling on-the-fly video sequenceswithout need of recording scenes in advance.
Revised: The modelling of dynamic scenes is formalized asan incremental learning paradigm which allows the method for handling on-the-fly video sequenceswithout need of recording scenes in advance.

Original: By virtue of narrow band tuning, our approach canachieve a fast training convergence.
Revised: By virtue of narrow band tuning, our approach canachieve a fast training convergence.

Original: The storage cost induced by using explicit grids can be obviouslyreduced by performing difference-base compression.
Revised: The storage cost induced by using explicit grids can be obviouslyreduced by performing difference-base compression.

Original: We also present an efficient training mechanismwith pilot model guidance to further improve model optimization.
Revised: We also present an efficient training mechanismwith pilot model guidance to further improve model optimization.

Original: Experiments demonstrate that ourapproach is capable of training a high-performing model for dynamic scenes with the speed of 10sfor tuning every frame, achievingover 1500 × speedup over the state-of-the-art implicit dynamicmethods.
Revised: Experiments demonstrate that ourapproach is capable of training a high-performing model for dynamic scenes with the speed of 10s fortuning every frame, achieving 1500 × speedup over the state-of-the-art implicit dynamic methods.

Original: As a future direction, we  expect to pursue further acceleration to support real-time trainingfor streaming dynamic scenarios.
Revised: On the other hand, we can still observe artifacts like translucent foreground and texture sticking in therendering images which may “inherit" from using an explicit representation. We expect to improvethe rendering quality by extending the framework with explicit-implicit hybrid representations whileremaining the benefit on training efficiency. Moreover, assigning adaptive bandwidth to differentparts by employing high-level understanding on scenes via extra supervisions, accelerating to supportreal-time training, or leveraging multiple key-frames would be valuable directions to improve theframework when applying it in real-word scenarios with extremely long sequences.