Original: Streaming Radiance Fields for 3D Video Synthesis
Revised: Streaming Radiance Fields for 3D Video Synthesis

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We present an explicit-grid based method for efficiently reconstructing streamingradiance fields for novel view synthesis of real world dynamic scenes.
Revised: We present an explicit-grid based method for efficiently reconstructing streaming radiance fields for novel view synthesis of real world dynamic scenes.

Original: Instead oftraining a single model that combines all the frames, we formulate the dynamicmodeling problem with an incremental learning paradigm in which per-framemodel difference is trained to complement the adaption of a base model on thecurrent frame.
Revised: Instead of training a single model that combines all the frames, we formulate the dynamic modeling problem with an incremental learning paradigm in which per-frame model difference is trained to complement the adaption of a base model on the current frame.

Original: By exploiting the simple yet effective strategy of tuning with narrowbands, the proposed method achieves a feasible framework for handling on-thefly video sequences with high training efficiency.
Revised: By exploiting the simple yet effective strategy of tuning with narrow bands, the proposed method achieves a feasible framework for handling on-thefly video sequences with high training efficiency.

Original: The storage overhead inducedby using explicit grid can be significantly reduced by using model differencebased compression.
Revised: The storage overhead induced by using explicit grid can be significantly reduced by using model difference based compression.

Original: We also introduce a curriculum learning strategy to furtheraccelerate model optimization for each frame.
Revised: We also introduce a curriculum learning strategy to further accelerate model optimization for each frame.

Original: Experiments on challenging videosequences demonstrate that our approach is capable of achieving a training speedof 10 seconds per-frame with competitive rendering quality, which attains 1500 ×speedup over the state-of-the-art implicit methods.
Revised: Experiments on challenging video sequences demonstrate that our approach is capable of achieving a training speed of 15 seconds per-frame with competitive rendering quality, which attains 1000 × speedup over the state-of-the-art implicit methods.

Original:  Introduction
Revised:  Introduction

Original: 3D video synthesis aims to realize free-viewpoint photo-realistic rendering for dynamic scenes,which are typically recorded via a set of cameras (with known poses) from multiple views and hasattracted much research effort due to its potential value in a wide range of applications in VR/AR.
Revised: 3D video synthesis aims to realize free-viewpoint photo-realistic rendering for dynamic scenes, which are typically recorded via a set of cameras (with known poses) from multiple views. The topic has attracted much research effort because of its potential value in a wide range of applications in VR/AR.

Original: Traditional techniques like mesh [1, 2], multi-sphere image [3] and depth [4] estimation are usuallyused for modeling and representing dynamic scenes.
Revised: Traditional techniques by estimating surfaces [1], multi-sphere images [2] or depth [3, 4] via multi-view stereo [5, 6] are usually used for modeling and representing dynamic scenes.

Original: However,the arbitrary geometry and complexappearance exhibited in real-world scenarios pose challenges for using a general methodology topursue high-performance modeling.
Revised: However, arbitrary geometry and complex appearance exhibiting in real-world scenarios pose challenges for leveraging a general methodology to pursue high-performance modeling.

Original: Neural radiance fields (NeRF) [5, 6] have recently emerged as a new methodology for effectivelyreconstructing and rendering static scenes via neural networks.
Revised: Neural radiance fields (NeRF) [7, 8] have recently emerged as a new methodology for effectively reconstructing and rendering static scenes via neural networks.

Original: By learning a continuous mappingbetween 3D points (given view directions) and its correspondingRGB radiance color and opacityequipped with volumetric rendering techniques [7], these methods have shown compelling ability forrendering high-fidelity images while encountering the bottleneck of  costly training and inference asrelying on a tremendous amount of computations through  networks.
Revised: These methods learn a continuous mapping between 3D points (given view directions) and their corresponding radiance colors and opacity, to realize high-fidelity rendering results through the use of volumetric rendering techniques [9]. However, they often suffer from costly training and inference due to a tremendous amount of computations through neural networks.

Original: When extending the paradigm toa dynamic scene, time steps are embedded as additional input [8] to achieve time-dependent rendering.
Revised: When extending the implicit formulation to a dynamic scene [10], time steps are embedded as additional input for training models across frames and achieving time-dependent rendering.

Original: However, the training overheadlinearly increases  with time, leading to enormous training costs whendealing with long sequences (e.g., costing about 56 GPU days for training 300 frames).
Revised: Compared to training on static scenes, training overhead increases significantly with respect to sequence length, e.g., costing about 56 GPU days for training 300 frames, that would be prohibitive when handling long sequences.

Original: The implicitformulation essentially suiting offline processing may not be easily generalized to handling onlinevideo sequences.
Revised: More importantly, such learning paradigm is restricted to offline modeling, thus unable to tackle online scenarios.

Revised: These novel view results are rendered at interactive speed ( ∼ 10 FPS) by StreamRF.

Original: In this work, we propose a novel method for reconstructing streaming radiance fields for novel view synthesis of 3D videos.
Revised: In this work, we propose a novel method for reconstructing streaming radiance fields for novel view synthesis of 3D videos.

Original: We formalize the modeling of a dynamic scene in an incremental learningThese novel view result are rendered at interactive speed ( ∼ FPS) by framework, enabling it to tackleon-the-fly video sequences in a streaming manner.
Revised: We formalize the modeling of a dynamic scene in an incremental learning framework, enabling it to tackle video sequences on-the-fly.

Original: Inspired by therecent success achieved by Plenoxel [9] on training and inference efficiency, our method is built onlearning of explicit grid representations.
Revised: Inspired by the recent success achieved by Plenoxel [11] on training and inference efficiency, our method is built on explicit grid representations.

Original: Specifically, the overall training objective is decomposedto complete training of a basic voxel grid model at the first frame with an incrementally tuning onmodel difference in subsequent frames. Only the model differences are expected to be trained andstored for each  frame.
Revised: Specifically, the overall training process is decomposed to the complete training of a basic voxel grid model at the first frame, with an online tuning mechanism for learning and storing model difference for each subsequent frame.

Original: A complete grid model given a time step i is simply attained by adding themodel difference to the previous model at i − 1 (or accumulating all the model differences before iwith the basic grid model).
Revised: A complete grid model given a time step i can be simply attained by adding the model difference to the previous model at i − 1 (or accumulating all the model differences before i with the basic grid model).

Original: By exploiting the prior of exhibiting local continuity between adjacentframes, we present a narrow-band tuning strategy for capturing and training the model differenceefficiently, associated with a compression strategy for significantly reducing memory cost whichmay plague most explicit-grid methods.
Revised: By exploiting the prior of exhibiting local continuity between adjacent frames, we present a narrow-band tuning strategy for capturing and training the model difference efficiently, associated with a compression strategy for significantly reducing memory cost that explicit grid methods struggle with.

Original: To further improve the training efficiency, we introducea curriculum learning paradigm for learning pilot models to guide the optimization of full-scalemodels.
Revised: To further improve the training efficiency, we propose to learning pilot models to guide the optimization of full-scale models.

Original: The overall framework is simple yet efficient.
Revised: The overall framework is simple yet efficient.

Original: Quantitative and qualitative experimentalresults validate the effectiveness of our approach.
Revised: Quantitative and qualitative experimental results demonstrate the effectiveness of our approach.

Original: The approach can achieve the training speed of 10seconds and inference time of 100 ms per frame for rendering a 1k resolution image with competitivequality, achieving 1,500 times speedup over the state-of-the-art dynamic neural radiance field methods(costing 56 GPU-days for training a sequence with 300 frames).
Revised: The approach can achieve the training speed of 15 seconds and inference time of 120 ms per frame for rendering a 1k resolution image with competitive quality, achieving three orders of magnitude faster than the state-of-the-art dynamic neural radiance field methods (costing GPU-days for training a sequence with 300 frames).

Original: It is also able to  obtain 135 timesacceleration with a much lower storage cost compared to per-frame explicit-grid benchmarks.
Revised: Compared to the efficient explicit-grid baseline by per-frame training, it still can obtain two orders of magnitude acceleration with a much lower storage cost.

Original: Theempirical analysis also demonstrates that usingsuch a proper incremental training mechanism wouldnot degradethe performance in the scenarios of dealing with relatively long sequences. 
Revised: The empirical analysis further show that using a proper incremental training mechanism would not degrade performance when dealing with relatively long sequences. code and dataset will be released at https://github.com/AlgoHunt/StreamRF

Original:  Problem Formulation
Revised:  Problem Formulation

Original: Neural Radiance fields (NeRF) [5] model a static scene implicitly by learning a continuous functionmapping from a 3D spatial position x ∈ R 3 specified in a viewing direction d ∈ R 3 (represented as Cartesian unit vectors) to RGB radiance color c ∈ R 3 and opacity σ ∈ R .
Revised: Neural Radiance fields (NeRF) [7] model a static scene implicitly by learning a continuous mapping from a 3D spatial position x ∈ R 3 specified in a viewing direction d ∈ R 3 (represented as Cartesian unit vectors) to RGB radiance color c ∈ R 3 and opacity σ ∈ R .

Original: The mapping functionis typically implemented by  multilayer perceptron (MLP)networks, f m : ( x , d ) (cid:55)→ ( c , σ ) , trainedwith a set of images with known camera pose.
Revised: The mapping function is typically implemented by a multilayer perceptron (MLP) f m : ( x , d ) (cid:55)→ ( c , σ ) , trained on a set of images with known camera pose.

Original: Given a ray r casting on the image plane through apixel, by using a classical volume rendering technique [10], the expected color ˆ C ( r ) of the pixel isrendered by integrating the opacity and the color of sampling points along the ray:
Revised: Given a ray r casting on the image plane through a pixel, by using a traditional volume rendering technique [12], the expected color ˆ C ( r ) of the pixel is rendered by integrating the opacity and the color of sampling points along the ray:

Original: [Equation]
Revised: [Equation]

Original: where δ i denotes the distance to adjacent points.
Revised: where δ i denotes the distance to adjacent points.

Original: These methods are subjected to slow training andrendering speed, since  ray casting involved in both training and rendering stage requires hundredsof forward propagation (and backward propagation in training) throughthe MLPs for the sampling points on each ray.
Revised: These methods are subjected to slow training and rendering speed, since the ray casting involved in both training and rendering stage requires hundreds of forward propagation (and backward propagation in training) through MLPs for the sampling points on each ray.

Original: For example, JaxNeRF[11], though optimized through parallelization, needs over20 seconds to render  with at least one day of training.
Revised: For example, JaxNeRF [13], though optimized through parallelization, needs overseconds to render an image with 800 × 800 resolution with over 10 hours of training.

Original: When extending the mechanism to model a dynamic scene, one straightforward solution is to embedthe time variable t into the representation, i.e., f t : ( x , d , t ) (cid:55)→ ( c , σ ) , where the training timewould linearly increase  with respect to t .
Revised: When extending the mechanism to model a dynamic scene, one straightforward solution is to embed the time variable t into the representation, i.e., f t : ( x , d , t ) (cid:55)→ ( c , σ ) , while training time would dramatically increase when dealing with a large number of frames.

Original: Training a video with  NeRF frameworks may cost over2,000 GPU-days.
Revised: Training a video with original NeRF frameworks may cost over 600 GPU-days.

Original: Although some variants, e.g., N3DV [8], improve training efficiency by virtueof sophistical designs,  it still needs about 56 GPU-days in training to obtain a model with reliablerendering quality.
Revised: N3DV [10] improves training efficiency by virtue of sophistical designs, however it still needs about 56 GPU-days for pursuing a model with reliable rendering quality.

Original: More importantly, these implicit formulations are naturally suitable for offlineprocessing which feeds all the (key)frames in advance for training, and maybe not  easily generalizedto online training scenarios.
Revised: More importantly, these implicit formulations are naturally suitable for offline processing which feeds all the (key)frames in advance for training, and may not be easily generalized to online training scenarios.

Original: On the other hand, recent progress, representative by Plenoxel [9], has demonstrated compellingimprovement on training and render efficiency for modelling a static scene.
Revised: Recent progress on explicit representations has demonstrated compelling improvement on training and render efficiency for modelling a static scene.

Original: Specially, a sparse voxelgrid is leveraged to replace the implicit representations in NeRF so that the scene is representedexplicitly.
Revised: The scene space is discretized and represented explicitly with a sparse voxel grid instead of learning representations implicitly with large MLPs. As a representative work, Plenoxel [11] optimizes voxel grids without neural networks.

Original: Each voxel in the grid consists of the scalar opacity σ and the coefficient vector k ofspherical harmonics (SH) for  describing  color information, which are directly optimized withoutneural networks.
Revised: Each voxel in the grid consists of the scalar opacity σ and the coefficient vector k of spherical harmonics (SH) for respectively describing geometric and color information.

Original: The color at the voxel v is obtained by,
Revised: The color at the voxel v is obtained by,

Original: [Equation]
Revised: [Equation]

Original: where ξ is the sigmoid function and { Y ml } denotes the set of SH basis.
Revised: where ξ is the sigmoid function and { Y ml } denotes the set of SH basis.

Original: To render the expected colorof the pixel through a ray, the opacity and the color of a sampling point x is calculated by performingtrilinear interpolation among the voxels within its neighborhood N ( x ) ,
Revised: To render the expected color of the pixel through a ray, the opacity and the color of a sampling point x are calculated by performing trilinear interpolation among the voxels within its neighborhood N ( x ) ,

Original: [Equation]
Revised: [Equation]

Original: v c ) .
Revised: [Equation]

Original: Then the pixel color is produced by integrating the sampling points along the ray based on Eq.
Revised: Then the pixel color is produced by integrating the sampling points along the ray based on Eq.

Original: Thevoxels with zero opacy (or truncated by a minimal threshold) are pruned after sparsification, and anoccupancy mask M with binary values is used for indicating if the space is occupied.
Revised: The voxels with zero opacity (or truncated by a minimal threshold) are pruned after sparsification, and an occupancy mask M with binary values is used for indicating if the space is occupied.

Original: More detailscan refer to the paper of Plenoxels [9].
Revised: More details can refer to the paper of Plenoxels [11].

Original: In theory, the mechanism could significantly reduce the overallcomputational complexity as it omits the requirement of numerous MLP-based costly computationduring training and inference.
Revised: In theory, the mechanism could significantly reduce the overall computational complexity as it omits the requirement of numerous MLP-based costly computation during training and inference.

Original: It is able to train a static scene with good rendering quality within 15minutes on single GPU and render a 1 k resolution image in 100 ms.
Revised: It is able to train a static scene with good rendering quality within 15 minutes on single GPU and render a 1 k resolution image in 100 ms.

Original: These efficiency gains pose thechance to develop a highly efficient framework for modeling and rendering a dynamic scene.
Revised: These efficiency gains would benefit developing a highly efficient framework for modeling and rendering a dynamic scene.

Original:  Streaming Radiance Fields for Dynamic Scenes
Revised:  Streaming Radiance Fields for Dynamic Scenes

Original: To realize sparse grids for the modeling of a dynamic scene, one intuitive way is to directly extend 3Dspatial dimensions (denoted by H × W × D ) with the 4-th dimension indicating  time  t for frames.
Revised: To realize sparse grids for the modeling of a dynamic scene, one intuitive way is to directly extend 3D spatial dimensions (denoted by H × W × D ) with the 4-th dimension indicating the time steps t for frames.

Original: However, using explicit grids for a static scenemay have faced the challenge on storage, althoughprogressive training and sparsifying (i.e, pruning off empty voxels) [9] are used to alleviate the issueof costly storage.
Revised: However, using explicit grids for a static scene have faced challenges on storage although progressive training and sparsifying (i.e, pruning off empty voxels) [11] are used to alleviate the issue of costly overhead.

Original: Simply expanding the grid into a space-time manner would make the storageoverhead increase linearly with t , which is infeasible for handling videos with long sequences.
Revised: Simply expanding the grid into a space-time manner would make storage overhead increase linearly with t , which is infeasible for handling videos with long sequences.

Original: Onthe other hand, per-frame training individually without considering temporal correlation may befeasible for achieving a tradeoff between storage and efficiency, while it is still far from the objectiveof pursing a high-performing  3D video synthesis framework.
Revised: On the other hand, per-frame training individually without considering temporal correlation may be feasible for achieving a tradeoff between storage and efficiency, while it is still far from the objective of pursing a high-performing framework for 3D video synthesis.

Original: To this end, we propose a novel method, named StreamRF, that realizes a streaming radiance field fordynamic scene rendering.
Revised: In this regard, we propose a novel method, named StreamRF, to realize a streaming radiance field for effectively rendering dynamic scenes.

Original: Our approach is built on the benefits of utilizing an explicit representationvia sparse voxel grids.
Revised: Our approach is built on the benefits of utilizing an explicit representation via sparse voxel grids.

Original: Particularly, by leveraging some intrinsic priors exhibiting in videos, wedevelop an incremental training framework which enables a faster training convergence and lowerstorage over per-frame grid modelling.
Revised: Particularly, by leveraging some intrinsic priors exhibiting in videos, we develop an incremental training framework which enables a faster training convergence and lower storage over per-frame grid modelling.

Original: Videos of forward-facing scenarios are typically composed of time-invariant components (e.g., staticbackground) and time-variant components (e.g., moving objects) with small variation exhibiting local continuity between adjacent frames.
Revised: Videos in forward-facing scenarios are typically composed of time-invariant components (e.g., static background) and time-variant components (e.g., moving objects) with small variation that exhibits local continuity between adjacent frames.

Original: By exploiting the fact, we decompose thesequentialmodeling problem  into two sub-problems.
Revised: By exploiting the fact, we decompose the problem of sequential modeling into two sub-problems.

Original: We first fully train a base grid model given the firstframe, denoted as V 

Revised: We first fully train a base grid model given the first frame, denoted as V

Original: = { V 0 c , V 0 σ } which stores the corresponding color features (the coefficients ofspherical harmonics) and opacity values.
Revised: = { V 0 c , V 0 σ } which stores the corresponding color features (the coefficients of spherical harmonics) and opacity values.

Original: Then the model is adapted to succeeding frames throughthe use of an effective tuning strategy.
Revised: Then the model is adapted to succeeding frames through the use of an effective tuning strategy.

Original: Per-frame tuning is expected to perform only on the changesbetween adjacent frames.
Revised: Per-frame tuning is expected to perform only on the changes between adjacent frames.

Original: The grid model in the current frame is produced by adding the modeldifference, which is optimized and stored during training phase, to the grid model in the previousframe.
Revised: The grid model in the current frame is produced by adding the model difference, which is optimized and stored during training phase, to the grid model in the previous frame.

Original: Formally, assume the model difference between the i − 1 -th and i -th frames is denoted asδ i =
Revised: Formally, assume the model difference between the i − 1 -th and i -th frames is denoted as δ i =

Original: ∆ ( V i , V i − 1 ) .
Revised: ∆ ( V i , V i − 1 ) .

Original: In the inference phase, the grid model at the time step i can be obtained by,
Revised: In the inference phase, the grid model at the time step i can be obtained by,

Original: [Equation]
Revised: [Equation]

Original: Such a incremental formulation indeed presents an online training paradigm, enabling the proposedframework to handleon-the-fly video sequences.
Revised: Such an incremental formulation indeed presents an online training paradigm, enabling the proposed framework to handle video sequences on-the-fly.

Original: 3.1 Tuning with Narrow Bands
Revised: 3.1 Tuning with Narrow Bands

Original: A problem arises: how to effectively capture the model difference between adjacent frames i − 1and i .
Revised: A problem arises: how to effectively capture the model difference between adjacent frames i − 1 and i .

Original: Given the sparse grid model V i − 1 , any object motion at the time step i may incur changes onthe opacity or colors of a part of voxels.
Revised: Given the sparse grid model V i − 1 , any object motion at the time step i may result in changes on the opacity and colors of a part of voxels.

Original: It is hence likely to happen that informative points occurin the originally empty space as well as some voxelsthat are originally occupied becomes empty.
Revised: It is hence likely to happen that informative points occur in the originally empty space as well as some voxels originally occupied becomes empty.

Original: Directly tuning the sparse grid can onlychange or one-way prune the non-empty voxels, intractableto handling long sequences.
Revised: If we directly tune the sparse grid, only the non-empty voxels could be changed or pruned in a one-way manner while unable to add informative voxels.

Revised: It is problematic when handling long sequences (shown by the ablation study on the sparse grid in Fig. 2).

Original: To address the issue, we propose a tuning strategy based on narrowbands.
Revised: To address the issue, we propose a tuning strategy based on narrow bands.

Original: We first retrieve and activate the narrow band for  frame i , i.e., the region of interest in the grid
Revised: We first retrieve and activate the narrow band for the frame i , i.e., the region of interest in the grid

Original: R i , based on the occupancy mask M i − 1 with binary values which indicates non-empty voxels at theframe i − 1 ,
Revised: R i , based on the occupancy mask M i − 1 (with binary values) which indicates non-empty voxels at the frame i − 1 ,

Original: [Equation]
Revised: [Equation]

Original: where F di and F er correspond to dilation and erosion operations, and ρ d and ρ e denote the corresponding radius parameters.
Revised: where F di and F er correspond to dilation and erosion operations, and ρ d and ρ e denote the corresponding radius parameters.

Original: ⊕ denotes the element-wise xor operation on thetwo masks to extractthe additional region of the dilated area compared to the eroded one.
Revised: ⊕ denotes the element-wise xor operation on the masks to extract the additional region of the dilated area compared to the eroded one.

Original: The narrow band actuallyrepresents a thin area around the surface of scenes (or objects in a composite scene).
Revised: The narrow band actually represents a thin area around the surface of scenes (or objects in a composite scene).

Original: We use it fortwo purposes: (1) For every empty voxels fall into the band, we restore them to the grid and initializethe opacity and color features with zero values.
Revised: We use it for two purposes: • For every empty voxel that falls into the band, we restore them to the grid and initialize the opacity and color features with zero values.

Original: (2) Only the voxels within the band are tuned and therest ones(with a large number) are not involved during the training stage for the frame i . 
Revised: • Only the voxels within the band are tuned and the rest ones are not involved during the training stage for the frame i . In other words, the tuning always performs on a relative small set of voxels in the frame i .

Original: We alsoprune empty voxels  to get a compact grid
Revised: We also prune empty voxels off to get a compact grid

Original: V i with an occupancy mask M i after tuning in the frame i .
Revised: V i with an occupancy mask M i after tuning.

Original: The key idea ofthe narrow band-based tuning is built upon leveraging the intrinsic priors that motionsinduced by moving objects typically show small changes and local continuity within neighboringareas between adjacent frames.
Revised: The key idea of narrow band-based tuning is built upon leveraging the intrinsic priors: motions induced by moving objects typically show small changes and local continuity within neighboring between adjacent frames.

Original: These assumptions have been successfully exploitedin a widerange of video processing [12] and  understanding tasks [13, 14], e.g, traditional mean-shift methodfor tracking [15].
Revised: These assumptions have been successfully exploited a wide of video processing [14] and video understanding tasks [15, 16], e.g, traditional mean-shift method for tracking [17].

Original: We demonstrate that the strategy can effectively benefit to training convergence,capable of achieving the speed ofper-frame ∼ 10 s  training for achieving a competitive quality (shownin section 5.4).
Revised: We demonstrate that the strategy can effectively benefit to training convergence and be capable of achieving the speed of ∼ 15 s per-frame training for achieving a competitive quality (shown in section 5.4).

Original: Compared to the state-of-the-art implicit method [8] with the training speed of [-4.
-] GPU-hours for each frame in average (costing about 56 GPU-days for training a sequence of frames), our method can achieve over 1,500 times speedup on training  over 200 times speedup on rendering efficiency (25 s VS 121 ms).
Revised: Compared to the state-of-the-art implicit method [10] with the training speed of [-4.
-]4.5 GPU-hours for each frame in average, our method is about three orders of magnitude faster on training and can achieve over 500 × speedup on rendering efficiency (over 60 s s VS 121 ms).

Original: 3.2 Difference-based Compression
Revised: 3.2 Difference-based Compression

Original: [Figure]

Original: Using explicit grid modelling yet poses the issue of large storage cost.
Revised: Using explicit grid modelling may pose the issue of large storage cost.

Original: Taking a grid size ( 1408 ×1156 × 128 in the experimental setting) for example, the overall storage  costs about 1015 MB aftersparsity for one-frame model, as a conventional sparse grid.
Revised: Taking a grid size ( 1408 × 1156 × 128 in the experimental setting) for example, the overall storage for a conventional sparse grid costs about 1015 MB after sparsity for one frame.

Original: By virtue of narrow band tuning, we onlyneed to store the model difference δ i with a significantly smaller size over the complete grid model.
Revised: By virtue of narrow band tuning, we only need to store the model difference δ i with a significantly smaller size compared to the complete grid model.

Original: The change of storage cost only happen in the following two scenarios, i.e., when adding voxels intothe grid (i.e., appearing in the empty space) and pruning voxels  from the grid (i.e., the previouslyoccupied space become empty).
Revised: The change of storage cost only happens in the following two scenarios, i.e., when adding voxels into the grid (i.e., appearing in the empty space) and pruning voxels off from the grid (i.e., the previously occupied space becomes empty).

Original: In this regard, we introduce a difference-based compression strategyto further reduce the storage overhead.
Revised: In this regard, we introduce a difference-based compression strategy to further reduce the storage overhead.

Original: Particularly, we can efficiently find out the change areas byusing the occupancy masks M i − 1 and M i between the adjacent frames i − 1 and i ,
Revised: We can efficiently find out the change areas by using the occupancy masks M i − 1 and M i between the adjacent frames i − 1 and i ,

Original: [Equation]
Revised: [Equation]

Original: 6) where M i a , M i e and M i r respectively denote the masks of adding voxels, erasing voxels and remainingones while the features at the voxels may change.
Revised: where M ia , M ie and M ir respectively denote the masks of adding voxels, erasing voxels and remaining ones while the features at the voxels may change.

Original: γ ( · ) defines the function of getting the L1 normalong the axis of color features (SH coefficients) on the model difference, and ξ denotes a threshold.
Revised: γ ( · ) defines the function of getting the L1 norm along the axis of color features (SH coefficients) on the model difference, and ϵ denotes a threshold.

Original: M ir actually involves the voxels with relatively large modification.
Revised: M ir actually involves the voxels with relatively large modification.

Original: Instead of storing model differenceδ i directly, we store the three masks, the voxel representations (SH coefficients and opacity) belongingto the adding area M ia and the representation differences for the voxels within the remaining area
Revised: Instead of storing model difference δ i directly, we store the three masks, the voxel representations (SH coefficients and opacity) belonging to the adding area M ia and the representation differences for the voxels within the remaining area

Original: M ir .
Revised: M ir .

Original: It can achieve a remarkable reduction rate (187 times) on storage that reduces 1015 MB to [-5.
-]
Revised: It can achieve a remarkable reduction rate (178 times) on storage that reduces 1015 MB to [-5.
-]5.

Original: MB for each frame in average.
Revised: MB for each frame in average.

Original:  Efficient Training with Pilot Model Guidance
Revised:  Efficient Training with Pilot Model Guidance

Original: To further improve the training efficiency of StreamRF, we exploit a curriculum learning paradigmthat trains from an easier problem.
Revised: To further improve the training efficiency of StreamRF, we exploit an optimization strategy inspired from curriculum learning to train from an easier problem.

Original: Formally, given the grid model V i − 1 , we adopt down-samplingoperations to get a pilot model of a smaller size with the scale S ,
Revised: Formally, given the grid model V i − 1 , we adopt down-sampling operations to get a pilot model of a smaller size,

Original: [Equation]
Revised: [Equation]

Original: When setting the S to 1 / 2 , the capacity of pilot model is roughly 1 / 8 of the full grid.
Revised: When setting the S to 1 / 2 , the capacity of pilot model is roughly 1 / 8 of the full grid.

Original: Training on asmaller model would reduce the computational complexity at every iteration (with the same batchsize) and also ease optimization.
Revised: Training on a smaller model would reduce the computational complexity at every iteration (with the same batch size) and also ease optimization.

Original: The model difference on the pilot models (cid:98) δ i =
Revised: The model difference on the pilot models (cid:98) δ i =

Original: ∆ ( (cid:98) V i , (cid:98) V i − 1 ) areimplemented by using { (cid:99) M ia , (cid:99) M ie , (cid:99) M ir } as following Eq. 6.
Revised: ∆ ( (cid:98) V i , (cid:98) V i − 1 ) are implemented by using { (cid:99) M ia , (cid:99) M ie , (cid:99) M ir } as following Eq. 6.

Original: Then we can get a guidance mask (cid:98) G i bymerging the three mask via elementwise "or" operator and upsampling to the full scale,[Equation]
Revised: Then we can get a guidance mask (cid:98) G i by merging the three mask via elementwise "or" operator and upsampling to the full scale,

Original: [Equation]
Revised: [Equation]

Original: The guidance mask is used for regularizing the gradient update when tuning the full-scale model.
Revised: The guidance mask is used for regularizing the gradient update when tuning the full-scale model.

Original: The values of added and erased voxels are filled back to the full-scale grid.
Revised: The values of added and erased voxels are filled back to the full-scale grid.

Original: All the voxels outside themask are frozen without optimization when training at the frame i .
Revised: All the voxels outside the mask are frozen without optimization when training at the frame i .

Original: The design of the strategy comes from the insight that training at a small size converges much fasterand the induced difference of pilot model indicates the voxels that need to be modified.
Revised: The design of the strategy comes from the insight that training at a small size converges much faster and the induced difference of pilot model indicates the voxels that need to be modified.

Original: Using thisstrategy can prevent unnecessary modification in the full-scale model and avoid the loss of highfrequency details during downsample-upsample process.
Revised: Using this strategy can prevent unnecessary modification in the full-scale model and avoid the loss of high-frequency details during downsample-upsample process.

Original: We observe that using this strategy facilitatepursuing a higher fidelity model compared to directly training on the full-scale grid with the sametraining time.
Revised: We observe that using this strategy facilitates pursuing a higher fidelity model compared to directly training on the full-scale grid with the same training time.

Original: Specially, the qualitative results (in Fig. 4) reveals that the model via directly trainingon the full-scale grid may encounter flicker issue when rendering along time steps to a sequence, i.e.,sparkling noise may occur in the stable background area, when training a short period.
Revised: The qualitative results (in Fig. 4) reveals that the model via directly training on the full-scale grid may encounter flicker issues when rendering along time steps, i.e., sparkling noise may occur in the stable background area when training period is short.

Original: The issuecould be addressed by lengthening the training schedule with a small learning rate.
Revised: The issue could be addressed by lengthening the training schedule with a small learning rate.

Original: While trainingwith the pilot model guidance can effectively prohibit flicker artifacts from happening, enabling amore efficient and stable optimization for the streaming paradigm.
Revised: While training with the pilot model guidance can effectively prohibit flicker artifacts from happening, enabling a more efficient and stable optimization for the streaming paradigm.

Original:  Experiments 
Revised:  Experiments 5.1 Datasets

Original: 5.1 Datasets

Original: Meet room Dataset. We build a multi-view capture system using 13 Azure Kinect cameras.
Revised: Meet Room Dataset We build a multi-view capture system using 13 Azure Kinect cameras.

Original: Weuse twelve cameras for training and one for testing, with an external pulse signal to synchronize allcamera shutter.
Revised: We use twelve cameras for training and one for testing, with an external pulse signal to synchronize all camera shutter.

Original: Though this system is capable of depth capture, we do not use depth results in thisresearch.
Revised: Though this system is capable of depth capture, we do not use depth images in this research.

Original: Thecapture videos were recorded at a resolution of 1280 × 720 and a frame rate of [-
-] FPS.
Revised: The videos are recorded at a resolution of 1280 × 720 and a frame rate of [-
-]30 FPS.

Original: The total size of the capture system was 100 cm × 75 cm, making it much sparser than somecommon datasets such as LLFF and N3DV.
Revised: The total size of the capture system is 100 cm × 75 cm and the captured views are aligned much sparser than some common datasets such as LLFF and N3DV.

Original: We will release this dataset for research purposes.
Revised: We will release this dataset for research purposes.

Original: Neural3D Video Dataset[8] contains 6 dynamic indoor scenes with varying illuminations, viewdependent effects, and highly volumetric effects.
Revised: Neural 3D Video (N3DV) Dataset [10] It contains 6 dynamic indoor scenes with varying illuminations, view-dependent and highly volumetric effects.

Original: Videos were captured at a resolution of 2704 × 2078and a frame rate of 30 FPS.
Revised: Videos are captured at a resolution of 2704 × 2078 and a frame rate of 30 FPS.

Original: Following the N3DV paper, we use one view for testing and 18 views fortraining. We choose the designated sequence and downsample it to 1352 × 1039 for a fair comparisonof the quantitative score.
Revised: Following the setting in the original paper [10], the quantitative score is produced on the designated sequence (“flame_salmon") by downsampling it to 1352 × 1039 for a fair comparison, where the frames captured by 18 views are used for training and the rest one for testing.

Original: 5.2 Implementation Details
Revised: 5.2 Implementation Details

Original: Initialization.
Revised: Initialization.

Original: We follow the standard training protocol in a forward-facing scene of Plenoxels[9].
Revised: We follow the standard training protocol of a forward-facing scene in [11] to obtain the base model at the first frame.

Original: We start by  256 × 256 × 128 grid, upsample and sparsify for every 38.4K iterations until the modelreaches the final resolution of 1408 × 1156 × 128 .
Revised: We start by a 256 × 256 × 128 grid which is upsampled and sparsified for every 38.4K iterations until the model reaches the final resolution of 1408 × 1156 × 128 .

Original: The total training takes 128K iterations with abatch size of 5,000 rays.
Revised: The total training takes 128K iterations with a batch size of 5K rays.

Original: We adopt the RMSProp [16] optimizer with a decay parameter ofβ = 0 .
Revised: We adopt the RMSProp [18] optimizer with a decay parameter of 0 .

Original: 95for voxel grid optimization.
Revised: 95 for optimization.

Original: Wealso adopt a higher TV loss to mitigatethe foggy issue, with λ TVset to 5 × 10 − 4 for opacity and 5 × 10 − 3 for color features.
Revised: We adopt a slighter larger TV penalty to mitigate foggy issues, where λ TV is set to 5 × 10 − 4 for opacity and 5 × 10 − 3 for color features.

Original: Per Frame Tuning.
Revised: Per Frame Tuning.

Original: The main model and the pilot model are trained in parallel.
Revised: The full-scale model and the pilot model are trained in parallel.

Original: We train the mainmodel with 500 iterations and  the pilot model with 1,000 iterations.
Revised: For the experiments on the Meet Room dataset, we train the pilot model with 1000 iterations and then tune the full-scale model with 500 iterations. For the experiments on the N3DV dataset, we train the pilot model with 750 iterations and then tune the full model with 500 iterations.

Original: We use RMSprop to optimizeboth models with a batch size of 20,000 rays.
Revised: We use RMSprop to optimize both models with a batch size of 20k rays.

Original: To promote the pilot model to have a better convergence,we enlarge λ TV  10 times when tuning the pilot model while keeping it untouched [Table]
Revised: To promote the pilot model to have a better convergence, we enlarge λ TV by 10 times when training the pilot model.

Original: Diff Based Compression.
Revised: Diff Based Compression.

Original: We set the SH coefficient threshold ξ to  1/27 (27 is the channel number of
Revised: We set the SH coefficient threshold ϵ to 1.5/27 and 1/27 for Meet Room

Original: SH coefficients).
Revised: Dataset and N3DV dataset respectively by the consideration of coefficient dimension (i.e., 27). We [Table]

Original: A smaller choice would drastically increase saving size with only marginal gain in
Revised: found using too small value would drastically increase saving size while the gain of rendering quality is marginal.

Original: PSNR.
Revised: In the experiments we use them by default to achieve a proper trade-off between storage cost and rendering quality.

Original: We convert all saving SH coefficient and density value to float16 to further decrease savingsize. In actual implementation we could merge M ia and M ie to one single mask to save more space.
Revised: In order to further reduce storage cost, we convert all saving SH coefficients and opacity to float16 and merge M ia and M ie to one single mask to save more space in the implementation.

Original: Moreover, as the mask is mostly  consecutive 0 and 1, both masks have a very high compression ratio(over 99%) even with  standard compression library ( e.g. ZLIB[17]).
Revised: Moreover, as the mask is mostly comprised of consecutive binary values, it can be compressed with a very high compression ratio (over 99%) simply with a standard compression library ( e.g. ZLIB[19]).

Original: We pack all tensors together andcompress them with ZLIB to get the final on-disk per frame storage.
Revised: We pack all tensors together and compress them with ZLIB to get the final on-disk storage cost for each frame.

Original: Pilot Model Guidance.
Revised: Pilot Model Guidance.

Original: We first downsample the grid model and apply the standard training pipeline(narrow band + difference-based compression) for pilot model training.
Revised: We first downsample the grid model and apply the standard training pipeline (narrow band and difference-based compression) for pilot model training.

Original: The induced model differenceis used in two purposes: 1) the values of added and erased voxelsare filled back to the full-scalemodel. 2) the guidance mask is used to regularize gradient update for tuning the full-scale model.
Revised: The induced model difference is used in two purposes, including filling the values of added and erased voxels back to the full-scale model and tuning the full-scale model with the guidance mask.

Original: 5.3 Results
Revised: 5.3 Experimental Comparison

Original: We first quantitatively compare  with benchmark methods in Table 1.
Revised: We first quantitatively compare the method with some representative works including N3DV which achieves the state-of-the-art rendering quality.

Original:  We use peak signal-to-noiseratio (PSNR),per frame trainingtime, and inference timeas evaluation metrics. We follow the N3DVtesting protocol [8] and choose “flame_salmon" as the test sequence. We evaluate the four methodsevery ten frames to calculate the mean value of per-frame metrics.
Revised: For the rest baselines, i.e., JaxNeRF [13], LLFF [20] and Plenoxels [11] which are originally designed and trained on static scenes, we train them in a per-frame independent manner, following the default setting as in their original papers. We use peak signal-to-noise ratio (PSNR), training and inference time and storage cost as metrics for a comprehensive evaluation, and report the mean scores per-frame.

Original: We refer to the values of PSNRand training time reported inthe paper of N3DV  for a fair comparison. We estimate the renderingtime of N3DV according to its network design.
Revised: We refer to the scores reported in N3DV [10] for a fair comparison and estimate the rendering time of the method according to its network design.

Original: Because they use a larger MLP than the oneused in NeRF [5], its inference time is supposed to be longer than NeRF.
Revised: As it used a larger MLP than the one in NeRF [7], its inference time is supposed to be longer than NeRF.

Original:  As shown in the table, thetwo implicit methods JaxNeRF and N3DV reveal high computation costthat requires over 8 GPU hours and 4 GPU hours for per-frame [-training (N3DV costs about 
 GPU days to train 300 frames),-] respectively.
Revised: All the baselines are evaluated every ten frames as in [10]. As shown in the Table 1 , the implicit methods JaxNeRF and N3DV present fairly high computation cost, requiring over 8 GPU hours and 4 GPU hours for per-frame [-training (N3DV costs about 
 GPU days to train 300 frames),-]training, respectively.

Original: As for testing speed, the LLFF achieve the fastestrecord.
Revised: As for testing speed, the LLFF achieve the fastest record.

Original: Because it only needs blending the multi-plane image to render a novel view.
Revised: Because it only needs blending the multi-plane image to render a novel view.

Original: However, itfails to produce a satisfactory result.
Revised: However, it fails to produce a satisfactory result.

Original: We achieve the optimization speed of costing 10 seconds only,outperforming all the baseline methods in terms of training efficiency.
Revised: We achieve the optimization speed of costing 15 seconds only, outperforming all the baseline methods in terms of training efficiency.

Original: Meanwhile our method canachieve a comparable PSNR to the previous SOTA.
Revised: Meanwhile our method can achieve a comparable PSNR to the previous SOTA.

Original: For the meet room dataset, we use the samesequence as in Fig 2 for testing and following the same test protocol as above.
Revised: For the meet room dataset, we use the same sequence as in Fig 2 for testing and following the same test protocol as above.

Original: Because our datasetcontains fast motion and camera spacing is larger than N3DV dataset, we could see a clear PSNRdrop on all four methods.
Revised: Because our dataset contains fast motion and camera spacing is larger than N3DV dataset, we could see a clear PSNR drop on all four methods.

Original: Nevertheless, our method still achieves a comparable performance withper frame training counterpart, around 0.3 drop in PSNR compared to JaxNeRF and Plenoxels,while obtaining the acceleration over 2 , 700 × and 80 × , respectively.
Revised: Nevertheless, our method still achieves a comparable performance with per frame training counterpart, around 0.4dB drop in PSNR compared to JaxNeRF and Plenoxels, while obtaining the acceleration over 2 , 700 × and 80 × , respectively.

Original: More experimental results andanalysis can be referred to supplementary materials.
Revised: More experimental results and analysis can be referred to supplementary materials.

Original: 5.4 Ablation study
Revised: 5.4 Ablation Studies

Original: Narrow Band Finetune We compare the narrow band with the following two counterparts, bothquantitative and visually: (1) directly adopt the original sparse voxels grid, (2) switch the originalsparse voxels grid to its dense counterpart.
Revised: Narrow Band Finetune. We compare the narrow band with the following two counterparts, both quantitative and visually: (1) directly adopt the original sparse voxels grid, (2) switch the original sparse voxels grid to its dense counterpart.

Original: It can be seen in Fig.2.
Revised: It can be seen in Fig.2.

Original: The sparse grid failed immediately(Fig.2 (a)), though the right person only move at a small distance.
Revised: The sparse grid failed immediately [Figure] (Fig.2 (a)), though the right person only move at a small distance.

Original: This artifact is because the subjecthas moved out of the trainable area.
Revised: This artifact is because the subject has moved out of the trainable area.

Original: A dense voxel grid allows every voxel to be trainable.
Revised: A dense voxel grid allows every voxel to be trainable.

Original: It still[Figure] fails when the right person has a large movement(Fig.2 (b)).
Revised: It still fails when the right person has a large movement(Fig.2 (b)).

Original: This artifact could be caused by therapidly increasing number of active voxels after switching from sparse to dense (5% vs. 100% of allvoxels).
Revised: This artifact could be caused by the rapidly increasing number of active voxels after switching from sparse to dense (5% vs. 100% of all voxels).

Original: Both PSNR curve and rendering result show our method achieves comparable PSNR with adense grid and can handle large movement in a dynamic scene.
Revised: Both PSNR curve and rendering result show our method achieves comparable PSNR with a dense grid and can handle large movement in a dynamic scene.

Original: Moreover, in this ablation study, ourmethod trains around 40% faster and requires 63% less memory consumption than dense finetune.
Revised: Moreover, in this ablation study, our method trains around 40% faster and requires 63% less memory consumption than dense finetune.

Original: Diff-based Compression We validate the performance and effectiveness of diff-based compressionin Fig.
Revised: Diff-based Compression. We validate the performance and effectiveness of diff-based compression in Fig.

Original: In this ablation, we first train our StreamRF on an entire sequence.
Revised: In this ablation, we first train our StreamRF on an entire sequence.

Original: The trained StreamRFis then saved with diff-based compression enabled and disabled.
Revised: The trained StreamRF is then saved with diff-based compression enabled and disabled.

Original: We then compare the PSNR andsaving size between the two test view sequences.
Revised: We then compare the PSNR and saving size between the two test view sequences.

Original: We present the per frame PSNR to show whetherdiff-based compression impairs the performance in the long run.
Revised: We present the per frame PSNR to show whether diff-based compression impairs the performance in the long run.

Original: The quantitative result shows that our compression method has negligible impact after a very longpropagation.
Revised: The quantitative result shows that our compression method has negligible impact after a very long propagation.

Original: Besides deducting overall storage size from 1015 MB to an average of 5.4 MB foreach frame, we can observe that performance of our method does not downgrade with time and theincremental storage for each frame is roughly stable along time steps.
Revised: Besides deducting overall storage size from 1015 MB to an average of 5.7 MB for each frame, we can observe that performance of our method does not downgrade with time and the incremental storage for each frame is roughly stable along time steps.

Revised: More details of model size reduction can be referred to supplementary materials.

Original: Pilot Model Guidance We validate our pilot model guidance visually in Fig.
Revised: Pilot Model Guidance. We validate our pilot model guidance visually in Fig.

Original: For base setting(right), we train the pilot model with 1,000 iterations and the full model with 500 iterations.
Revised: For base setting (right), we train the pilot model with 1,000 iterations and the full model with 500 iterations.

Original: For theablation setting (left), we directly train the full model with 1,000 iterations to make per frame trainingtime equivalent in both settings.
Revised: For the ablation setting (left), we directly train the full model with 1,000 iterations to make per frame training time equivalent in both settings.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4, pilot model guidance help reduce various kindsof artifacts: (top) vanishing hands and blurry elbow.
Revised: 4, pilot model guidance help reduce various kinds of artifacts: (top) vanishing hands and blurry elbow.

Original: (bottom) sparkle noise in a white wall, the fuzzyarea between two people.
Revised: (bottom) sparkle noise in a white wall, the fuzzy area between two people.

Original: These results verified our intention that we could use an easy-to-optimizedsmall model to provide a more optimal initialization to the full model.
Revised: These results verified our intention that we could use an easy-to-optimized small model to provide a more optimal initialization to the full model.

Original:  Related Work
Revised:  Related Work

Original: Static Novel View Synthesis Novel view synthesis from a set of input images with known cameraposes has been studied intensively in the field of computer vision and computer graphics.
Revised: Static Novel View Synthesis Novel view synthesis from a set of input images with known camera poses has been studied intensively in the field of computer vision and computer graphics.

Original: Earlierworks like Lumigraph[19, 20] and Light-Fied [21, 22]  focus on interpolating the sampled light raysfor realistic rendering.
Revised: Earlier works like Lumigraph [21, 22] and Light-Field [23, 24] focus on interpolating the sampled light rays for realistic rendering.

Original: It has been proved [23, 24] that the sampling density heavily depends on theunderlying geometry of the captured scene.
Revised: It has been proved [25, 26] that the sampling density heavily depends on the underlying geometry of the captured scene.

Original: Thus, many works strive to recover an approximate [25,26] or precise geometry model [27] to facilitate novel view rendering.
Revised: Thus, many works strive to recover an approximate [27, 28] or precise geometry model [29] to facilitate novel view rendering.

Original: Among those works, multiplane image (MPI) [18, 28–30] is a promising approximate geometry model that enables realisticrendering of generic scenes.
Revised: Among those works, multiplane image (MPI) [20, 30–32] is a promising approximate geometry model that enables realistic rendering of generic scenes.

Original: More recently, NeRF [5] represents the radiance field with a multi-layer perceptron (MLP) networkand combines it with differentiable volume rendering for view interpolation.
Revised: More recently, NeRF [7] represents the radiance field with a multi-layer perceptron (MLP) network and combines it with differentiable volume rendering for view interpolation.

Original: This approach achievedtremendous success and there are many variants to push its limits towards every aspect.
Revised: This approach achieved tremendous success and there are many variants to push its limits towards every aspect.

Original: Theworks [6, 31–33] focus on improving rendering quality, while the works [34–36] strive to acceleraterendering speed, and other works [9, 37–40] propose to replace implicit function by an explicit orhybird counterpart to boost training convergence.
Revised: The works [8, 33–35] focus on improving rendering quality, while the works [36–38] strive to accelerate rendering speed, and other works [11, 39–42] propose to replace implicit function by an explicit or hybird counterpart to boost training convergence.

Original: Dynamic Novel View Synthesis Novel view synthesis has also been heavily studied with dynamicscenes.
Revised: Dynamic Novel View Synthesis Novel view synthesis has also been heavily studied with dynamic scenes.

Original: Early effort of generating 3D video can trace back to the 90s [1, 41], which computes explicitdepth maps to facilitate novel view interpolation.
Revised: Early effort of generating 3D video can trace back to the 90s [4, 43], which computes explicit depth maps to facilitate novel view interpolation.

Original: This approach has been combined with moresophisticated capturing hardware to generate streamable free-viewpoint video in [2].
Revised: This approach has been combined with more sophisticated capturing hardware to generate streamable free-viewpoint video in [1].

Original: Matusik and
Revised: Matusik and

Original: Pfister [42] further integrate camera arrays and 3D displays to build an end-to-end 3D TV system.
Revised: Pfister [44] further integrate camera arrays and 3D displays to build an end-to-end 3D TV system.

Original: More recent works learn a neural network to capture radiance fields of dynamic scenes.
Revised: More recent works learn a neural network to capture radiance fields of dynamic scenes.

Original: A familyof methods [43–49] realize view synthesis by training on monocular videos captured by a movingcamera, i.e., viewpoints change with respect to time steps.
Revised: A family of methods [45–51] realize view synthesis by training on monocular videos captured by a moving camera, i.e., viewpoints change with respect to time steps.

Original: The works in [46–48] decouple theoverall radiance field reconstruction to the learning of a static canonical template and a deformationfield warping to the template.
Revised: The works in [48–50] decouple the overall radiance field reconstruction to the learning of a static canonical template and a deformation field warping to the template.

Original: Park et al.
Revised: Park et al.

Original: [49] lifts canonical templates into hyper-spaces to induceflexibility on modelling variations in topology.
Revised: [51] lifts canonical templates into hyper-spaces to induce flexibility on modelling variations in topology.

Original: Several methods model dynamics through the useof scene flows [43, 45] or depth prior [44], therefore they highly reply on the regularization ofextra supervision (e.g., optical flow, depth or foreground-background segmentation estimated bywell-trained models).
Revised: Several methods model dynamics through the use of scene flows [45, 47] or depth prior [46], therefore they highly reply on the regularization of extra supervision (e.g., optical flow, depth or foreground-background segmentation estimated by well-trained models).

Original: Broxton et al.
Revised: Broxton et al.

Original: [3] and Attal et al.
Revised: [2] and Attal et al.

Original: [50] use multi-camera stereo rig and extend
Revised: [52] use multi-camera stereo rig and extend

Original: MPI to MSI to support view interpolation with a larger field of view.
Revised: MPI to MSI to support view interpolation with a larger field of view.

Original: Bansal et al.
Revised: Bansal et al.

Original: [51] use multiplemobile phones to capture a dynamic scene and compose the static and dynamic components withneural networks for 4D space-time visualization.
Revised: [53] use multiple mobile phones to capture a dynamic scene and compose the static and dynamic components with neural networks for 4D space-time visualization.

Original: The closest work to our setting is Neural 3D
Revised: The closest work to our setting is Neural 3D

Original: Video (N3DV)[8], which captures dynamic scenes with synchronous video sequences from multiplecameras.
Revised: Video (N3DV)[10], which captures dynamic scenes with synchronous video sequences from multiple cameras.

Original: It directly extends NeRF with an additional timecode input, and accelerates computationwith a hybrid importance sampling and hierarchical training.
Revised: It directly extends NeRF with an additional timecode input, and accelerates computation with a hybrid importance sampling and hierarchical training.

Original: N3DV can produce strong visual resultsgiven enough optimization, while it is very time consuming, requiring over per frame 5 GPU hours intraining and 10+ seconds in rendering, making it impractical for real-world applications.
Revised: N3DV can produce strong visual results given enough optimization, while it is very time consuming, requiring over per frame 5 GPU hours in training and 10+ seconds in rendering, making it impractical for real-world applications.

Original: On the other hand, some approaches adapt the advances in neural rendering for the modeling ofspecific objects, e.g., human bodies [52, 53] by integrating with domain-specific priors, e.g., skeletonor SMPL mesh [54].
Revised: On the other hand, some approaches adapt the advances in neural rendering for the modeling of specific objects, e.g., human bodies [54, 55] by integrating with domain-specific priors, e.g., skeleton or SMPL mesh [56].

Original: In contrast, our method imposes no prior in objects and is capable of representingmore general scenes with complex deformation.  
Revised: In contrast, our method imposes no prior in objects and is capable of representing more general scenes with complex deformation.  Limitation & Future work [Figure] In Fig.

Original:   Conclusion
Revised: 5 we can still observe artifacts like translucent foreground and texture sticking in the rendering images which may "inherit" from using an explicit representation. We expect to improve the rendering quality by extending the framework with explicit-implicit hybrid representations while remaining the benefit on training efficiency. Moreover, assigning adaptive bandwidth to different parts by employing high-level understanding on scenes via extra supervisions, accelerating to support real-time training, or leveraging multiple key-frames would be valuable directions to improve the framework when applying it in real-word scenarios with extremely long sequences.  Conclusion

Original: We propose a novel streaming radiance field method for effectively reconstructing and renderingdynamic scenes with explicit grid representations.
Revised: We propose a novel streaming radiance field method for effectively reconstructing and rendering dynamic scenes with explicit grid representations.

Original: The modelling of dynamic scenes is formalized asan incremental learning paradigm which allows the method for handling on-the-fly video sequenceswithout need of recording scenes in advance.
Revised: The modelling of dynamic scenes is formalized as an incremental learning paradigm which allows the method for handling on-the-fly video sequences without need of recording scenes in advance.

Original: By virtue of narrow band tuning, our approach canachieve a fast training convergence.
Revised: By virtue of narrow band tuning, our approach can achieve a fast training convergence.

Original: The storage cost induced by using explicit grids can be obviouslyreduced by performing difference-base compression.
Revised: The storage cost induced by using explicit grids can be obviously reduced by performing difference-base compression.

Original: We also present an efficient training mechanismwith pilot model guidance to further improve model optimization.
Revised: We also present an efficient training mechanism with pilot model guidance to further improve model optimization.

Original: Experiments demonstrate that ourapproach is capable of training a high-performing model for dynamic scenes with the speed of 10s fortuning every frame, achieving 1500 × speedup over the state-of-the-art implicit dynamic methods.
Revised: Experiments demonstrate that our approach is capable of training a high-performing model for dynamic scenes with the speed of 15s for tuning every frame, achieving 1000 × speedup over the state-of-the-art implicit dynamic methods.

Original: On the other hand, we can still observe artifacts like translucent foreground and texture sticking in therendering images which may “inherit" from using an explicit representation.

Original: We expect to improvethe rendering quality by extending the framework with explicit-implicit hybrid representations whileremaining the benefit on training efficiency.

Original: Moreover, assigning adaptive bandwidth to differentparts by employing high-level understanding on scenes via extra supervisions, accelerating to supportreal-time training, or leveraging multiple key-frames would be valuable directions to improve theframework when applying it in real-word scenarios with extremely long sequences.