D-MiSo: Editing Dynamic 3D Scenes using Multi-Gaussians Soup

Authors: joanna  waczy
piotr  borycki
joanna  kaleta
sławomir  tadeja
przemysław  spurek
Venue: NeurIPS_cc_2024_Conference
Published: 2024-04-25 23:32:07

Abstract: Over the past years, we have observed an abundance of approaches for modeling dynamic 3D scenes using Gaussian Splatting (GS). These solutions use GS to represent the scene's structure and the neural network to model dynamics. Such approaches allow fast rendering and extracting each element of such a dynamic scene. However, modifying such objects over time is challenging. SC-GS (Sparse Controlled Gaussian Splatting) enhanced with Deformed Control Points partially solves this issue. However, this approach necessitates selecting elements that need to be kept fixed, as well as centroids that should be adjusted throughout editing. Moreover, this task poses additional difficulties regarding the re-productivity of such editing. To address this, we propose Dynamic Multi-Gaussian Soup (D-MiSo), which allows us to model the mesh-inspired representation of dynamic GS. Additionally, we propose a strategy of linking parameterized Gaussian splats, forming a Triangle Soup with the estimated mesh. Consequently, we can separately construct new trajectories for the 3D objects composing the scene. Thus, we can make the scene's dynamic editable over time or while maintaining partial dynamics.

Main paper: 
 introduction: 
 Recently introduced Gaussian Splatting (GS) [1] represents the 3D scene structure through Gaussian components. We can combine GS with the neural network (i.e., deform network) to model dynamic scenes [2] . This approach involves the joint training of both the GS components and the neural network. GS characterizes the 3D object's shape and color, while the neural network utilizes time embedding and Gaussian parameters to generate updated initial positions to model dynamic scenes. Such an approach allows for fast rendering and extracting each element of a dynamic scene.Most existing methods can effectively model dynamic scenes, but generating new 3D objects' positions remain challenging. Consequently, we cannot edit objects over time when using such Figure 2 : Each object using the D-MiSo model is represented by Core-Gaussians and Sub-Gaussians, which form Multi-Gaussians. Each Gaussian is related to a triangle using parameterization proposed in GaMeS [3] . Triangles define the Gaussian shape (i.e., location, scale, rotation), and triangles clouds form Triangles Soups. approaches. To tackle this issue, SC-GS (Sparse Controlled Gaussian Splatting) [4] uses Deformed Control Points to manage Gaussians. After the training phase, we can manually modify the model at any point in time. However, this method requires identifying elements to remain static and adjusting 3D objects' centroids (nodes) during editing when a relationship between the selected nodes is visible. For example, by moving the humanoid 3D model's hand, the part of the head or leg is also changed. To address this issue, we introduce Dynamic Multi-Gaussian Soup (D-MiSo), which is easier to modify (Fig. 1 ) and obtain renders comparable to SC-GS. D-MiSo estimates the mesh as Triangle Soup, i.e. a set of disconnected triangle faces [5, 6] , and uses a dynamic function to control the vertices. D-MiSo employing Multi-Gaussians, defined as larger Core-Gaussians encompassing smaller ones termed Sub-Gaussians (Fig. 2 ). Sub-Gaussians are defined in the local coordinate system given by principal components of Core-Gaussian. Therefore, by modifying Core-Gaussian, we change all Sub-Gaussians, which allows scene modifications (Fig 3 ). Core-Gaussians are an alternative to the control points discussed in [4] , with the added advantage of allowing individual modifications. Consequently, there is no necessity for static and dynamic markers. In Fig. 4 , we present the difference between modification applied by SC-GS and D-MiSo.Our model uses flat Gaussians. Therefore, based on GaMeS [3] , we can approximate Gaussian components using triangle face mesh by parameterizing Gaussian components by the vertices of the mesh face. We denote such transformation by T (•). In practice, as a consequence of parameterizing each Gaussian, we obtain a cloud of triangles called Triangle Soup [5] . Using Triangle Soup, we can control two types of Gaussian components. Accordingly, in D-MiSo, we get: Sub-Triangle Soup, Core-Triangle Soup and Multi-Triangle Soup (Fig. 2 , Core-Triangle Soup is marked by red color, and Sub-Triangle Soup is denoted in blue). In D-MiSo, we can select and modify one part of the object like a mesh. In contrast, using SC-GS, static and dynamic points have to be selected, and editing only one part of the object is difficult.During training, the positions of Core-Gaussians are managed by deformation multilayer perceptron (MLP), while the Sub-Gaussians are collectively manipulated through global transformation and small local deformation. The former describes the general flow of objects in the scene, while the local deformation is responsible for modeling small changes like shadows and light reflections. After training, we can modify our model directly by using the vertex of the Sub-Triangle Soup, or we can generate mesh from the Core-Triangle Soup (Fig. 5 ).The contributions of this paper are significant and are outlined as follows:• We introduce the Multi-Gaussian components, which consist of a single large Gaussian response for global transformations and many small components dedicated to rendering. Multi-Gaussian components allow for the modeling of large 3D scene elements.• We propose D-MiSo a model that uses Multi-Gaussian components and two deformation networks for modeling dynamic scenes. We also must add static (pink) and dynamic (yellow) points in SC-GS to obtain modification by editing dynamic points.In practice, we have to use many static points to stop artifacts. Moreover, SC-GS is not an affine invariant and produces space when we change the size of the objects. In the case of D-MiSo, we marked points and applied modifications. Our model is superior in handling object scaling.• Our D-MiSo allows an object to be edited at a selected moment in time. The edited components are independent, and the editing does not affect other parts of the object. In addition, it also allows for full or partial dynamics to be maintained. Modifications also include scaling and rotation. 
related works: 
 Recent advancements in view synthesis, particularly driven by NeRFs [7] , have significantly contributed to the rapid development of novel view synthesis techniques. However, the majority of these approaches model static scenes implicitly using MLP. Moreover, several works have extended classical NeRF to dynamic scenes through the use of deformation fields [8, 9, 10] and [11] . The alternative approaches, such as [12] and [13] , represent scenes as 4D radiance fields. Early works on dynamic scenes face difficulties when dealing with monocular settings and uncontrolled or lengthy scenarios. To enhance scene motion modeling, some works utilize flow-based techniques [10, 14, 15] . However, NeRF-based solutions often suffer from long training and rendering times. To address this, grid-plane-based methods [16, 17, 18] have been proposed. In addition, several NeRF-based approaches have also been extended for scene editing purposes [19, 20, 21, 22] .The recently introduced Gaussian Splatting (GS) technique [1] addresses many limitations of other methods, offering multiple advantages due to their explicit geometry representation, enabling easier dynamics modeling. The efficient rendering of the 3D version of GS also avoids densely sampling and querying neural fields, making downstream applications such as free-viewpoint video reconstruction more feasible. A notable extension of 3D GS was proposed in [23] , where the authors introduced the concept of anchor points to tackle the problem of overfitting caused by redundant Gaussians. Scaffold-GS addresses this issue by distributing local 3D Gaussians according to anchor points. This approach reduces redundancy, enhances scene coverage, and maintains high-quality rendering with improved robustness to view changes. While the original GS was developed for static scenes, several extensions for dynamic scenes were proposed. Most of the early works operate in multiview setup [24, 25, 26] . For example, [26] utilizes a frame-by-frame approach to model each timestep. However, this method lacks inter-frame correlation and requires high storage overhead for long-term sequences.In [27, 2] , MLP is introduced to model changes in Gaussians over time, and in [28] MLP together with decomposed neural voxel encoding algorithm are utilized for training and storage efficiency. In [29] , dynamic scenes are divided into dynamic and static parts, optimized separately and rendered together to achieve decoupling. [24] spacetime Gaussian proposes approximating the spatiotemporal 4D volume of a dynamic scene by optimizing a collection of 4D primitives with explicit geometry and appearance modeling. This method uses 4D Gaussians parameterized by anisotropic ellipses and Figure 5 : One way to modify the object at the selected time t i is to take Core-Gaussians and apply a meshing strategy to obtain the correct mesh instead of Triangle Soup. Then, we can parametrize Sub-Gaussian in the coordinate system given bay mesh faces instead of Core-Triangle Soup. Finally, we can modify our mesh to obtain new modifications.view-dependent, time-evolved appearances represented by 4D spherical harmonics coefficients. In [25] , a novel, real-time and photorealistic scene representation called Spacetime Gaussian Feature Splatting has been introduced. This approach extends 3D Gaussians with temporal opacity and parametric motion/rotation, enabling the capture of static, dynamic, and transient scene content. Additionally, the method incorporates splatted feature rendering to model view-and time-dependent appearances while maintaining a compact representation size. The method is notable for its high rendering quality and speed while also being storage-efficient. Other works enhance dynamic scene reconstruction using external priors. For example, the diffusion priors can be used as regularization terms during optimization [30] .Furthermore, GS was employed for mesh-based scene geometry editing. In [31] , 3D Gaussians are defined over an explicit mesh and utilize mesh rendering to guide adaptive refinement. This approach depends on the extracted mesh as a proxy and fails if the mesh cannot be extracted. In contrast, in [32] , explicit meshes are extracted from 3D GS representations by regularizing Gaussians over surfaces. However, this method involves a costly optimization and refinement pipeline. Another example of [33] employs sparse control points for 3D scene dynamics, but this method struggles with intense edit movements and necessitates accurate static node selection. Also, [3] combines GS with mesh extraction. However, such an approach only works for static scenes.Figure 6 : Multi-Gaussian, consisting of one Core-Gaussian G Vj and a Sub-Gaussian G i Vj . The Core Gaussian is parametrized by a V j -triangle, and the Sub-Gaussian by a V i j -triangle. The relative distance of the center of the Sub-Gaussian from the Core-Gaussian is indicated byα i α i α i = (α i 1 , α i 2 , α i 3 ).The method proposed in [34] combines meshes with GS and reconstructs a high-fidelity and time-consistent mesh from a single monocular video. However, it relies on a Poisson Solver and differentiable Marching Cubes to recover the deformed surface, significantly complicating the pipeline. Moreover, it does not explore geometry modification capabilities, which constitute a significant aspect of our work. Conversely, cage-based methods [35, 36] are an intuitive tool for geometry manipulation. However, they require additional steps for cage-building and may lack the flexibility and precision of manual, vertex-level deformation techniques, potentially missing fine details.In contrast to the listed approaches, we propose a D-MiSo, a mesh-based method specifically designed to handle dynamic scenes. D-MiSo leverages a straightforward pipeline of GS techniques to enable real-time editing of dynamic scenes.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The paper clearly presents both technical as well as ethical limitations in a dedicated paragraph.Guidelines:• The answer NA means that the paper has no limitation, while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
dynamic multi-gaussian soup: 
 Here, we present the main components of D-MiSo. We start with the classical GS to provide the foundations for our model. Next, we introduce the concept of Multi-Gaussians and describe how to estimate the mesh for editing. Finally, we show D-MiSo, which uses Multi-Gaussians in dynamic 3D scenes.Gaussian Splatting The Gaussian Splatting (GS) technique models 3D scenes using an array of 3D Gaussians, each specified by its mean position, covariance matrix, opacity, and color expressed using spherical harmonics (SH) [37, 38] . The GS algorithm constructs the radiance field by iteratively optimizing the parameters of all Gaussian components. Ultimately, the GS efficiency mainly depends on its rendering method, which involves projecting Gaussian components.The GS framework employs a dense collection of 3D Gaussians:G = {(N (m i , Σ i ), σ i , c i )} n i=1, where m i denotes the position, Σ i the covariance, σ i the opacity, and c i the SH colors for the i-th Gaussian. The GS optimization process involves a repetitive cycle of rendering and comparing the resultant images with the training views. In our work, we will use Multi-Gaussian approaches (Fig. 6 ). Multi-Gaussians Multi-Gaussians G multi are dedicated to describing relatively large parts of the 3D scene to allow modification of large blocks instead of modifying each small Gaussian separately. The Multi-Gaussian model comprises a primary large 3D Gaussian (referred to as the Core-Gaussian G core ), which encompasses numerous smaller Gaussians (termed Sub-Gaussians G sub ), all of which are parameterized by the main Core-Gaussian. Multi-Gaussianare is similar to anchor Gaussians from [39] , but we do not use a neural network to produce child components. We parametrize Sub-Gaussians in a local coordinate system.Similarly to classical GS, we parameterize the Core-Gaussian distribution by center m and the covariance parameterized by factorization: Σ = RSSR T , where R is the rotation matrix and S the scaling parameters. More precisely we consider p Core-Gaussians uses flat Gaussians as in [3] , and is defined by:EQUATIONwhere S = diag(s 1 , s 2 , s 3 ), s 1 = ε and R is rotation matrix of Core-Gaussian defined as: R = (r 1 , r 2 , r 3 ), where r i ∈ R 3 which can be interpreted as a local coordinate system used by Sub-Gaussian.It is worth noting that Sub-Gaussian can be interpreted as a child of Core-Gausian. We define centers of Sub-Gaussian N sub (m i , R i , S i ) in the local coordinate system of Core-Gaussian N core (m, R, S) by: m i = m + Rα i α i α iT , where m, R is Core-Gaussian position and rotation; andα i α i α i = (α i 1 , α i 2 , α i3 ) are trainable parameters used to define the positions of the Sub-Gaussian relative to the Core-Gaussian (Fig. 6 ). Sub-Gaussians are used for rendering and can be seen as a main component of our model:EQUATIONwhere m, R, S are parameters of Core-Gaussian and, α i , S i , R i marks parameters of i-th Sub-Gaussians with opacity σ i , and SH colors c i .Core-Gaussians is generally dedicated mainly to transformations, hence, in practice opacity or colors are not used during rendering. On the other hand, Sub-Gaussian is devoted to rendering and modification. It has its own opacity and colors.Figure 9 : One way to animate is to assign the nearest triangle (face) to each Sub-Gaussian from the estimated mesh. The mesh modification changes the assigned Gaussian.Multi-Gaussians describe 3D scenes using solid blocks rather than tiny Gaussian distributions. This method enhances our model's suitability for dynamic environments. One of our mode's most important properties is its ability to model dynamic scenes in each time step. To obtain such properties, we parameterize all Gaussian using Triangle Soup following similar approach as in GaMeS [3] . Thanks to a few simple transformations, we can convert the mean m rotation R and scaling S into triangle V = (v 1 , v 2 , v 3 ), which parametrizes Gaussian distribution. Such transformation is unambiguous and reversible.Let us assume that we have a Gaussian component parameterized by mean m, rotation matrix R = [r 1 , r 2 , r 3 ] and scaling S = dig(ε, s 2 , s 3 ). We define three vertex of a triangle (face): V = [v 1 , v 2 , v 3 ],, where v 1 = m, v 2 = m + s 2 r 2 , v 3 = m + s 3 r 3 .V = [v 1 , v 2 , v 3 ]and reparameterize the Gaussian component by defining m, R = [r 1 , r2 , r3 ] and Ŝ = dig(ŝ 1 , ŝ2 , ŝ3 ). First, we put m = v 1 . The first vertex of R is given by a normal vector: where × is the cross product. The second one is defined by r2 = (v2-v1) ∥(v2-v1)∥ . The third one is obtained as a single step in the Gram-Schmidt process [40] :r1 = (v 2 -v 1 ) × (v 3 -v 1 ) ∥(v 2 -v 1 ) × (v 3 -v 1 )∥ ,r3 = orth(v 3 -v 1 ; r 1 , r 2 ).Scaling parameters can also be easily calculated ass 1 = ε, ŝ2 = ∥v 2 -v 1 ∥ and ŝ3 = ⟨v 3 -v 1 , r3 ⟩.Consequently, the covariance of Gaussian distribution positioned on face is given by: ΣV = RV ŜV ŜV RT V , and correspond with the shape of a triangle V . For one face (v 1 , v 2 , v 3 ), we define the corresponding Gaussian component: Finally, the Gaussian component is derived from the mesh face parameters. This approach can be applied in a Multi-Gaussian framework. Therefore, we will use invertible transformation between Gaussian parameters and triangle face T and notation:N ((v 1 , v 2 , v 3 )) = N ( mV , RV , ŜV ).N (V ) := N (T -1 (V )) = N ( mV , RV , ŜV ).In our D-MiSo we parameterize the p Core-Gaussians with k Triangle soup:EQUATIONwhere V j , α i α i α i , R i , Ŝi , σ i , c i are trainable parameters and ( mVj , RVj , ŜVj ) = T -1 (V j ). Alternatively, we can parameterize Core-Gaussian and Sub-Gaussians by Triangle Soup:EQUATIONwhere( mVj , RVj , ŜVj ) = T -1 (V j ), V i = T ( mV + RV α i α i α iT , Ri , Ŝi ).In D-MiSo, we use the collation of Multi-Gaussian distribution for rendering and Sub-Triangle Soup for editing. The formal definition of our model uses equation (3) since, in training, we store Core-Gaussian as a triangle face (Core-Gaussian does not have colors) and Sub-Gaussian as a collection from classical GS components with color and opacity. After training, we parametrize our model to equation (4) for editing.Previously, we defined Multi-Gaussians and their parametrization using Triangle Soup. Now, we have all the tools to present the D-MiSo model. The overview of our method is illustrated in Fig. 7 .The input to our model is a set of images of a dynamic scene, together with the time label and the corresponding camera poses. Our training is divided into two stages. In the first stage, we initialize the Core-Gaussians. In the second, we add Sub-Gaussian components.Stage 1 First, we train only Core-Gaussians to obtain good initialization for Multi-Gaussins.As Core-Gaussians are mainly employed to capture motion, our model only requires their small amount (Fig. 7 ). In our approach, the Core-Triangle Soup, constructed via the Core-Gaussians parameterization (as depicted in Fig. 2 ), is adjusted depending on the time t.In practice, when random initialization of Gaussians is necessary, redundant Gaussians must be pruned first to ensure that the remaining ones represent the object's shape. To reduce the number of Gaussians and obtain consistent Core-Gaussians, we train GS on a batch containing a few views instead of one.In practice, we render a handful of views from different positions and use back-propagation.In particular, we parameterize Gaussians N (m, R, S) by faceV = (v 1 , v 2 , v 3 ) to ob- tain N (v 1 , v 2 , v 3 ).Our Deform Network takes as in input the triangle vertices V assigned to the parameterized 3D Core-Gaussians and the current time t and returns updated ψ(V, t) = (∆v 1 (t), ∆v 2 (t), ∆v 3 (t), ∆R V (t)). Such updates consist of translation and rotation (Fig. 8 ):V (t) = V ⊙ ψ(V, t) = (v 1 + ∆v 1 (t), v 2 + ∆v 2 (t), v 3 + ∆v 3 (t)) • ∆R V (t).D-MiSo parameterized Core-Gaussians in time t by: N core (V (t), σ, c) = N core (V ⊙ ψ(V, t), σ, c) where ψ is a deformable network that moves triangle V according to time t. The opacity and color of the Core-Gaussian are used only in the first stage. The position mi V (t) = mV (t) + RV (t) α i α i α iT of the Sub-Gaussian in time t is determined by the position mV (t) , and rotation RV (t) of the Core-Gaussian (parameterized by triangle V ) and the learning parameter α i α i α i . It should be noted that scale S i , color c i , and opacity σ i of Sub-Gaussian are trainable and do not depend on time. Sub-Rot Network produce updated ϕ(R i , t) = ∆R i (t) for rotation parameter of Sub-Gaussians. Finally parameters of Sub-Gaussians in time t depends on Deform Network ψ, and Sub-Rot Network ϕ, and the Corr-Gaussians parameter V , and Sub-Gaussian parameters R i , S i , c i and σ i :G Sub (t) = {(N sub ( mV ⊙ψ(V,t) + RV ⊙ψ(V,t) α i α i α iT , R i + ϕ(R i , t), S i ), c i , σ i )} k i=1 .D-MiSo final model consists of two deformable networks and two levels of Gaussian distributions. This approach enables efficient modeling of object motion over time in dynamic scenes, allowing for adjustments to objects at each time frame.The result of the model's inference depends on the camera's view angle as well as on the selected time. Only Sub-Gaussian features are used in the rendering process. Hence, the implementation of generating an output image from Gaussians is no different from a vanilla GS.The modifications involve the appropriate transformation of Sub-Gaussians. We base this on the fact that the transformed Gaussians have exactly the same color properties as before editing. Only the shape of the Gaussians is changed. In our work, we present three editing methods (Fig. 11 ). The experimental section is divided into two parts. First, we show that D-MiSo can model dynamic scenes with high quality; in the second part, we demonstrate that our model allows an easy editing procedure, which is our main contribution.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
reconstruction of dynamic scenes: 
 Here, we outline the specifics of our implementation and provide a detailed description of the used datasets. We demonstrated the core advantages of our model by conducting experiments using three different datasets. The source code is available on 3 . Our code is developed on top of the GS vanilla code, according to their license. We used NVIDIA GeForce RTX 4090 and A100 GPUs. The experiments focus on a benchmark task of reconstruction. PSNR metric is used to compare our model with other methods. Furthermore, additional numerical comparisons, i.e., SSIM/LPIPS, are available in the supplementary material.D-NeRF Datasets: Contains seven dynamic objects with realistic materials described with a single camera [11] . This means the model had access to only one view at a given moment. Tab. 1 shows we are very comparable to other methods, and we achieve a higher PSNR on one object. The differences in metrics are small, however, our method presents an easier way to edit (Fig. 4 ). Providing, among other things, better scaling. Following previous methods, our results are obtained on images using 400 by 400 resolution with a black background.NeRF-DS [44] : This dataset contains again seven real-world scenarios containing a moving or deforming specular object. Each scene was recorded using two cameras, with the video captured by one of them being used as a training set, while the footage from the second one was treated as a test set. The camera pose was estimated using COLMAP for both cameras. Tab. 2 shows that our method achieves the SOTA results for distinct objects.PanopticSports Datasets: The dataset comprises six dynamic scenes featuring significant object and actor movements [26, 45] . The scenes are categorized by the activities performed in the video sequences, i.e., juggling, boxing, softball, tennis, football, and basketball. Each scene was recorded using 31 cameras over 150 timesteps. Following the official data split, we use footage from 27 cameras for training and the remaining 4 cameras for testing. Numerical results are shown in Tab. 3.The results show that the model achieved SOTA for five objects according to PSNR metrics and six objects according to LPIPS metrics.These results demonstrate that D-MiSo is comparable to other methods. Moreover, our main contribution in comparison to other methods is the very easy editing of the resulting object (Fig. 4 11).As mentioned earlier, we proposed three new methods of output object modification. The first one focuses on moving and preparing a formal mesh, which is connected in contrast to Triangle Soup. We apply a simple meshing strategy on Core-Gaussians. We used the basic Alpha Shape algorithm [46, 47] and showed that the mesh does not have to estimate the surface perfectly to be effective. Then, we reparametrize the Sub-Gaussian by finding the closest face from the mesh (Fig. 9 ).In practice, we represent each Sub-Gaussian in a local coordinate system (analogically to Multi-Gaussians). Therefore, each Sub-Gaussian is assigned to the nearest face (Fig. 9 ). Each face can have a different number of Sub-Gaussians prescribed in such a configuration. This method allows us to maintain the consistency bestowed by the mesh. The whole process is shown in Fig. 5 . Thanks to such representation, we can edit our connected mesh to produce the correct edition of the dynamic scene (Fig. 2 ). The second editing method allows us to define the connections, i.e., editing Sub-Triangles Soup directly, e.g. moving a hand or bending horns (Fig. 10 ) or rotating a human body (Fig. 3 ). Changes are possible because we can also transform a group of Sub-Triangles instead of individual ones. This method allows for even very subtle changes like raising the thumb (Fig. 10 ), turning the hand over (Fig 4) , opening or closing the hand (Fig. 12 ). These edits would be difficult in other approaches based on adjusting the 3D objects' centroids (nodes) since the space of nodes is limited in details area (Fig. 4 ). Nodes' use is preeminent for defining movement, which was insignificant in these places.With this method, we can also change complicated objects like a 360 • scene without losing the dynamics-related model. For example, the rotation of a person stacking boxes (Fig. 3 ). The third method focuses on transforming the space in which the object is located. Similar to the second method, the third one also works directly on the Sub-Triangle Soup. We can achieve such an effect by applying a certain function to the selected plane. In practice, the object, or a portion of it, is modified, for instance, through the use of sinusoidal functions. It allows us to obtain fluidity and more easily define the physical nature of the movement (Fig. 1 , 11, 13).Our methods are also scalable, so we can easily remove or duplicate elements from an image. Moreover, the duplicated elements can be given their own dynamics. Examples of these effects are shown in Fig. 13 , where we removed the plate and both duplicated and rescaled the blue balls multiple times. 
conclusion: 
 D-MiSo is a novel method based on Gaussian Splatting parameterization, which produces a cloud of triangles called Triangle Soup. The method allows easy editing of objects created in inference with the possible transformations, including moving, scaling, and rotating. By defining Multi-Gaussians, the obligatory separability of modified elements seen in other models is combated. In addition, certain elements of objects can be duplicated and removed (Fig. 13 ). Furthermore, the D-MiSo method can facilitate giving different dynamics to separate parts of an object (Fig. 1 ).Limitation The method allows for complex changes at a given moment in time. However, if some area is not well represented in the training set, it is impossible to edit them. For example, a person's hand can be changed but not fingers (Fig. 4 ). This is due to the liminality of Triangle Soup relative to a well-fitted mesh.Broader impact Our model significantly improves rendering quality and advances 3D scene reconstruction and rendering, impacting multiple domains by enabling more realistic and efficient 3D modeling and animation. This technology could enhance VR/AR experiences [48] , robotics [49] , and medical imaging [50] . It could also be used for interactive education [50] , scientific visualization, and a plethora of other commercial applications like product design and real estate [51] . 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: We cited all utilized works and resources according to the citing guidelines. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not relate new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing or research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: The paper does not involve crowdsourcing or research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: The detail-capturing capabilities of Sub-Gaussians can be effectively adapted to other GS-based approaches, particularly in cases where native methods lack dedicated support for high-detail reconstruction [52, 53, 54] or edition [55, 56] .Recent studies have investigated the integration of human faces, avatars, and meshes for 3D human modeling and rendering from video, presenting a promising avenue for enhancing detailed reconstruction.For instance, [57] introduces a human body representation where neural features across frames share a latent code anchored to a deformable mesh, facilitating cross-frame integration and efficient 3D representation learning in sparse monocular video scenarios. Similarly, HumanNeRF [58] optimizes a volumetric human representation in a canonical T-pose, using a decomposed motion field to handle both skeletal and non-rigid deformations. This method provides high-quality, free-viewpoint renderings in challenging settings. UV-Volumes [59] further address the high computational costs associated with NeRF, achieving real-time, editable video rendering by encoding appearance details into 2D neural texture stacks for efficient 3D density and texture estimation.Gaussian-based representations have gained traction for modeling dynamic avatars, as in [60, 61, 62, 63, 64] , with most approaches leveraging video data and human pose/face priors like FLAME fitting [65] . Building on these advances, we propose that D-MiSo concepts could be effectively adapted for dynamic motion modeling and small detail capture in avatar representations. Specifically, Stage 1 could focus on integrating such models for capturing fine-grained avatar motion with anchored details.This section contains a numerical comparison of the D-MiSo with other models regarding the experiments described in the main document. We used additional LPIPS and SSIM metrics. During the experiments, we used the RTX4090 GPU and densification until 5000 iterations (1st Stage). We see that we get comparable competitive results. We can see that our method achieves better results for five objects from the D-NeRF dataset in terms of the LPIPS metric (Tab. 4). Full numerical comparison with metrics for the NeRF-DS dataset shows that we are comparable to other methods in the reproduction task (Tab. 5). In our method, a batch of images is taken as input to the model. Tab. 6 shows a numerical comparison using batch: 4, 8 on D-NeRF. Training takes 80 thousand iterations, and the second stage starts at the 5 thousandth iteration. Each Core-Gaussian has attached 25 Sub-Gaussians. This is comparable to the SC-GS implementation. Tab. 6 also presents the training time and storage cost, as well as the FPS for rendering for each dataset. These results suggest that our model is memory efficient and that the time required to train it is minimal. With batch equal to 8 in all cases, storage costs decreased, and performance improved with a trade-off of increased training time.A similar batch study has been done for the NeRF-DS dataset presented in Tab.7. We can see that batch is playing a bigger role in the dataset, which is considered a more pronounced move.We contrasted our model with and without the Sub-Rotation Network. The Sub-Rot Network, despite being a simple single-layer network, plays a crucial role in the second stage. Tab. 8 shows a numerical comparison of PSNR for models with and without the Sub-Rot Network. Furthermore, we would like to emphasize that due to its shallow architecture, incorporating the Sub-Rot Network does not significantly impact training time. We decided to present the results on the jumpingjack dataset distinguishing between batches of 4, 8. These results show that the Sub-Rot Network is crucial to obtaining SOTA PSNR. With Sub-Rot Network, we obtain approximately 0.5 PSNR crucial to obtain the reconstruction of small elements (like human fingers in the jumpingjack dataset)Tab. 9 shows how the size of the network influences training time and render speed. These results show that deformable networks and Sub-Rot Networks are not costly in terms of rendering time, resulting in real-time rendering. In practice, batch size has a higher impact than deformation network depth. Our model operates on several hyperparameters, primarily derived from the basic GS framework. Given the introduction of additional stages and the multi-Gaussian component, one of the new hyperparameters is the number of Core-Gaussians and Sub-Gaussians (Tab. 10).The ablation study also focuses on the influence of the Sub-Rot Network on training time and PSNR (Tab. 8). Tables show training time and PSNR analysis.The number of Core-Gaussians is determined automatically during stage 1, utilizing the pruning mechanism implemented in GS. The Tab. 10 illustrates the training time corresponding to these parameters using the jumpingjacks dataset as an example with batch equal to 8. We can see the speed of the 1st stage, which is used for Core-Gaussian preparation. Moreover, too few Core-Gaussians can cause a drop in render quality (PSNR metric). The experiments suggest the number of Core-Gaussians is sufficient after the first phase. In all cases, a larger number of Sub-Gaussians improved the quality of renders. The third method of modification allows us to give new dynamics and fluidity to the object (Fig. 16 ). We can see that even difficult edits like bending the back, changing the shape of the face, and bending the hand are possible while the result is natural from the point of view of graphics. This is made possible by editing the scale and rotation of Gaussians in an explicitly defined way. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.https://github.com/waczjoan/D-MiSo 


Decision: This paper introduces a Dynamic Gaussian Splatting representation that enables easier object shape editing at test time. There is an overall consensus among reviewers that this is an interesting submission and should be accepted (2x Borderline Accept, 1x Weak Accept, and 1x Accept). Therefore, I recommend acceptance.

For the camera-ready version, please polish the paper to make it more reader-friendly and include the new experiments and discussions.

Reviews: review: paper_summary: paper_summary: This paper introduces a Dynamic Gaussian Splatting representation that allows for easier object shape editing at test time. This is achieved through the use of Dynamic Multi-Gaussian Soup (D-MiSo), a mesh-inspired multi-gaussian system. Specifically, a GS is divided into two components: Core-Gaussian, which models the transformation of groups of sub-gaussians, and sub-Gaussian, which handles the final geometry and color. This enables different levels of editing by either changing the core-Gaussian or sub-Gaussians for more fine-grained control.
strength_weakness: strength_weakness: I find the paper's writing style difficult to follow and some sentences do not even make sense. Additionally, there are numerous components involved in the pipeline that are challenging to keep track of: multi-Gaussian, core-Gaussian, sub-Gaussian, core-triangle soup, multi-triangle soup, and sub-Gaussian soup (which may be the same thing?). I would appreciate a revision in writing to improve clarity.
questions: questions: 1) Is there a strategy for selecting hyperparameters such as the number of Gaussian and Sub-Gaussian components?
2) It is unclear why the proposed method would perform worse than SC-GS on metrics like PSNR/SSIM for NeRF-DS and D-NeRF. Does this suggest that increased editability comes at the cost of model performance, or is it simply a matter of hyperparameter tuning?
limitations: limitations: The authors have discussed the limitations of their method, but I was unable to locate a statement regarding the broader impacts of the proposed method in the Limitations section, as claimed by the author.
paper_summary: paper_summary: This paper proposes a novel framework for modeling and editing dynamic scenes. The authors used a two-pass Multi-Gaussian approach to represent the entire scene. First, they obtained relatively stable Core Gaussians through initialization, and then used the Core Gaussians to drive Sub-Gaussians to fit the entire scene. To better edit motion, the authors parameterized each Gaussian with triangle soup. Experimental results show that this method not only matches the rendering quality of SC-GS but also enables the editing of more extreme large motions.
strength_weakness: strength_weakness: 1. I think the following papers should also be cited, because they have made significant contributions to the early dynamic scenes reconstruction based on Gaussian splatting.:
   - (CVPR 2024) Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis by Zhan Li et al.
   - (ICLR 2024) Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting by Zeyu Yang et al.
2. L136, The concept of anchor Gaussian originally comes from Scaffold-GS [1], not from Spec-Gaussian. It would be even better if Scaffold-GS could be cited.
3. It would be even better if Deformable-GS [2] were included in the comparison of quantitative metrics (such as Tabs. 1-2), as it is the first deformation-based dynamic Gaussian splatting method and serves as the baseline for SC-GS.
4. I think the ablation study is not thorough enough. Although the ablation on batch size is appreciated, I am unclear about the roles of other components of the method. Corresponding ablation studies are needed to make the paper more robust. For example, I am very interested in understanding the impact of the `Sub-Rot Network` mentioned in L218 on the rendering metrics.



[1] Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. 2023. Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering. arXiv preprint arXiv:2312.00109 (2023)

[2] Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv preprint arXiv:2309.13101,2023.
questions: questions: A small tip: In Tab. 1, the authors used the metrics from the SC-GS paper. However, SC-GS did not use a consistent background; for example, the `Bounce` (and maybe `Trex`) scene used a white background. The authors clearly used a black background consistently. Although the reported metrics in the table show a slight decrease compared to SC-GS, I greatly appreciate the authors' honesty.
limitations: limitations: Please refer to the weakness part.
paper_summary: paper_summary: This paper is developed based on SC-GS, which enhanced 3DGS with Deformed Control Points to model low-rank dynamics and modify dynamic objects over time. SC-GS necessitates selecting elements that need to be kept fixed and centroids that should be adjusted throughout editing, and it poses additional difficulties regarding re-productivity editing. This paper proposes Dynamic Multi-Gaussian Soup (D-MiSo), which serves as a kind of mesh representation of dynamic GS, links parameterized Gaussian splats, forming a Triangle Soup with the estimated mesh, and separately constructs new trajectories for the 3D objects composing the scene, which makes the scene’s dynamic editable over time or while maintaining partial dynamics.
strength_weakness: strength_weakness: Lack of training time comparisons.

Lack of comparison against gs2mesh-based methods and cage-based methods.
questions: questions: Lack of discussion with respect to the human-NeRF (Neural Body/HumanNeRF/UV-Volumes) and human-GS methods (Animatable Gaussians/D3GA/GoMAvatar).
limitations: limitations: The authors adequately addressed the limitation.
paper_summary: paper_summary: This paper proposes the D-Miso pipeline, which includes multi-Gaussian components and two deformation networks to model global transformation and local deformation. A good training strategy is introduced to fit the proposed methods. The rendering quality looks good, and the editing results seem better than those of current motion-based sparse control point deformation using 3D Gaussians.
strength_weakness: strength_weakness: 1. Multi-Gaussian representations are similar to SC-GS's control points. Since 3D/4D Gaussians themselves can model large 3D scene elements, this is not a strength. I think the contribution is minor, and it seems that no ablation study is presented in the paper.
2. Most demos are provided in synthetic datasets, which is not convincing. Results on real-world datasets such as [9, 13] are recommended to be included.
3.The "Related Works" section should be reorganized. More discussions on multi-view dynamic scenes and flow-based radiance fields should be included. 
4. If your method is based on [3], a brief review should be included in the main text as a preliminary to improve the presentation.
5. More details such as storage costs and training time (for each stage) should be included. Additionally, would the rendering speed be affected by the large deformation network?
6. Minor issues: Line 140: "Guassinas" -> "Gaussians," Line 197: "vertixes" -> "vertices," Fig. 10: "possition" should be "position."
7. What is n_core as mentioned in Eq.1?
questions: questions: 1. More comparison and discussions with SC-GS should be included since SC-GS can also support editing and share the similar ideas with control points/Gaussians.
2. As I understand, deformation-based methods find it challenging to synthesize correct novel views with large motion. Could the authors provide comprehensive insight into this issue?
limitations: limitations: Yes, the limiations have been addressed.

Comments: title: Authors’ response
Authors’ response
Authors’ response
Authors’ response
Authors’ response
Authors’ response
Keep positive
comment: Thank you for dedicating your time to reviewing our manuscript. We believe that the revisions have given the work greater context, enhancing its value for the reader. We sincerely appreciate the decision to increase our score.
We appreciate the careful and constructive evaluation of our work and engaging in the discussion.
Thank you for your answer. All my concerns are addressed.
Thanks for the reply. It addressed my concerns. The progress from human-NeRF to human-GS should be discussed together, and at the same time, it should be more concluded and concise. Based on other reviews and the rebuttal, especially the training time comparisons and discussion against gs2mesh-based, cage-based, human-NeRF and human-GS methods, I raise my rating to Weak Accept.
It is a non-trivial question requiring substantial consideration; thank you for posting it. 

 - **D-MiSo vs SC-GS performance on synthetic data:**

The synthetic dataset from the paper describing D-NeRF [11] is the first and most explored dataset of dynamic 3D objects. **Modern methods on the D-NeRF dataset produce similar results with small differences caused mainly by parameter optimization.** Results on the D-NeRF dataset are proof that our model has attained the presently possible optimal level of rendering quality (Reviewer 2: “Experimental results show that this method not only matches the rendering quality of SC-GS but also enables the editing of more extreme large motions.”). **Consequently, our model's main contribution is better editing quality.**

- **D-MiSo vs SC-GS performance on real data:**
 
In the SC-GS paper, the authors discuss the use of control points. Each control point influences the closest Gaussian components. The number of control points is a crucial hyperparameter. As detailed in Section S1.1 of the SC-GS supplementary material, determining the **optimal** number of control points is essential for achieving accurate reconstructions. However, following the SC-GS supplementary material, we think that in real scenes with complex backgrounds, identifying the appropriate number of control points may be challenging. Increasing the number of control points does not necessarily lead to better performance due to optimization challenges. 

In contrast, our D-MiSo model automatically covers space, and no additional hyperparameters are introduced. The number of Core Gaussians is automatically found by the classical Gaussian Splatting optimization procedure in the first part of optimization. Later, each scene part can be modified independently.  As a result, our model not only achieves high-quality reconstructions but also facilitates more effective editing, thus balancing reconstruction quality with editing capability. In essence, our solution does not use additional parameters, and our model builds a natural division of objects by the Core Gaussian component. It also allows for more straightforward modification. **Consequently, our model's main contribution is better editing quality.** 

[11] Pumarola, Albert, et al. "D-NeRF: Neural radiance fields for dynamic scenes." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.
We thank the Reviewer for the feedback. To provide a more balanced discussion, aside from already included Gaussian Splatting-based 3D avatars, we will include in our manuscript all the suggested NeRF-based works recognized for their following contributions:

In [h1], authors introduce a novel human body representation, where learned neural representations at different frames share the same set of latent codes anchored to a deformable mesh. This approach allows for the natural integration of observations across frames and provides geometric guidance, leading to more efficient learning of 3D representations. Neural Body effectively addresses the challenge of ill-posed representation learning in scenarios with highly sparse monocular video views. Whereas HumanNeRF [h2] optimizes a volumetric representation of a person in a canonical T-pose, with a motion field that maps the canonical representation to each frame of the video via backward warps. The motion field is decomposed into skeletal rigid and non-rigid motions, handled by deep networks. HumanNeRF demonstrates significant performance improvements over previous methods and produces compelling free-viewpoint renderings from monocular video, even in challenging, uncontrolled capture scenarios. In addition, to address the high computational costs of NeRF rendering, [h3] proposes the UV-Volumes approach, which enables real-time, editable free-view video of human performers. By separating high-frequency appearance details from the 3D volume and encoding them into 2D neural texture stacks (NTS), the UV-Volumes allow for the use of smaller and shallower neural networks to achieve efficient 3D density and texture coordinate estimations while maintaining detailed 2D appearance capture. 

If this clarification adequately addresses the concerns raised, we kindly ask the Reviewer to consider raising their scores accordingly.

[h1] Peng et al. 2021, "Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans," Proceedings of the IEEE/CVF CVPR

[h2] Weng et al. 2022, "HumanNeRF: Free-Viewpoint Rendering of Moving People From Monocular Video," Proceedings of the IEEE/CVF CVPR

[h3] UV-Volumes - Chen et al. 2023, "UV Volumes for Real-Time Rendering of Editable Free-View Human Performance," Proceedings of the IEEE/CVF CVPR
Thanks for the rebuttal. It addressed my main concerns. However, I don't think the human-NeRF-based methods (such as Neural Body, HumanNeRF, and UV-Volumes) should be ignored. I believe it is fundamental for this paper to select some of these works and discuss the line of progress.
Thank you for your comprehensive answer. I am still curious why your method peforms worse on synthetic data which should be easier?
We thank the Reviewer for the constructive feedback and appreciate increasing the score.
We thank the Reviewer for the valuable comments and are keen to follow up on the provided suggestions:

- We will add information on image resolution and include a table with 800x800 resolution in the Appendix.
- We agree that KNN can be fixed and will include a comment to that effect in the paper. 

- We will discuss the differences and connections between D-MiSo and GaMeS in the main paper.
Thanks to the authors for their response. I am satisfied with the rebuttal and will maintain my positive evaluation. There are a few points to note for the release version:
- The resolution of Deformable-GS is 800x800, while SC-GS and D-MiSo are 400x400. I don't think it's necessary to align them completely, but it would be better to clarify this.
- For SC-GS FPS measurement, KNN should be fixed, and there is no need to query for every iteration.
- I suggest that the authors discuss the differences and connections between D-MiSo and GaMeS.
Thanks for your comprehensive answers, most of my questions are solved and would like to change my score to BA.