# Gaussian-UDSR: Real-Time Unbounded Dynamic Scene Reconstruction with 3D Gaussian Splitting
## 3 Method
### Gaussian Model
The following is the local approximation of the multivariate function at a point:  
\[J= \begin{bmatrix}\frac{df_{1}}{dx}&\frac{df_{1}}{dy}&\frac{df_{1}}{dz}\\ \frac{df_{2}}{dz}&\frac{df_{2}}{dy}&\frac{df_{2}}{dz}\\ \frac{df_{3}}{dx}&\frac{df_{3}}{dy}&\frac{df_{3}}{dz}\\ \end{bmatrix} = \begin{bmatrix}\frac{n_{1}}{z}&0&-\frac{n_{1}x}{z^{2}}\\ 0&\frac{n_{2}}{z}&-\frac{n_{2}y}{z^{2}}\\ 0&0&-\frac{n_{2}f}{z^{2}}\\ \end{bmatrix}  \tag{3}\]  
\[\Sigma^{\prime}=JW\Sigma_{s}W^{T}J^{T} \tag{4}\]  
where \(n_{1}\) is the focal length in the \(X\)-axis direction, \(n_{2}\) is the focal length in the \(Y\)-axis direction, and \(-\frac{nf}{z^{2}}\) is set to 0, disregarding the \(Z\)-axis direction; \(W\) is the transformation matrix from the world coordinate system to the camera coordinate system. Figure 3: Visualization of linear transformations of a sphere. Gaussians with confidence intervals more significant than 99% are retained based on the view cone, invalid Gaussians are eliminated, and the remaining Gaussians are quickly sorted based on the depth in the camera space, and then the attributes of each 2D Gaussian are queried for pixel-by-pixel rendering based on the point-based \(\alpha\)-blending:  
\[C_{s}=\sum_{i\in N}c_{si}\alpha_{si}\prod_{j=1}^{i-1}(1-\alpha_{sj}) \tag{5}\]  
where \(\alpha_{si}\) represents the opacity value of the current point \(i\), \(\alpha_{sj}\) represents the opacity value of each point before \(i\). We multiplied it with \(1-\alpha_{sj}\) as the color weight, representing that the more transparent were all the previous points \(j\), the greater the contribution of the color of the point \(i\) to the rendering. The scene semantics \(\beta\) and depth \(\gamma\) can be derived from the rendering, Equation (5). The adaptive density control module copies the Gaussian distribution of the under-reconstructed region and optimizes the direction and shape of ellipsoids based on gradient feedback, ensuring adequate region coverage. When expressing the scene, a large number of Gaussian distributions are superimposed to describe the scene area, and the Gaussian distribution cannot accurately express the scene if it is insufficiently reconstructed or over reconstructed. The adaptive density control module copies the Gaussian distribution of the under-reconstructed region into two Gaussian distributions and optimizes the Gaussian sphere according to the direction and shape of the returned gradient to ensure that the Gaussian ellipsoid can fill the region well. For the over-reconstructed region, the larger Gaussian distribution is first split into two, and then the two Gaussian ellipsoids are scaled down by the scaling factor, and similarly the position and shape of the Gaussian ellipsoid are optimized according to the returned gradient to ensure that the Gaussian ellipsoid can fill the region well. Dynamic object modeling. Dynamic objects interact with the environment differently at different moments, and their scaling matrix \(S_{d}\) and opacity \(\alpha_{s}\) are kept consistent with the static background model. The difference, however, is that the dynamic object's pose is defined in the object's local coordinate system. In order to transform it into the world coordinate system (i.e., the static background coordinate system) and introduce time \(t\)  
Figure 4: Visualization of Jacobian matrix in projection model. we propose the pose-tracking mechanism. Expressly, in the world coordinate system, we represent the change of the dynamic object's position in the time-flow field by a set of time-dependent translation vectors \(\{T_{t}\}\) and rotation matrices \(\{R_{t}\}\):  
\[\{(T_{t},R_{t})\}_{t=0}^{T}=\{(T_{0},R_{0}),(T_{1},R_{1}),(T_{2},R_{2}),\cdots, (T_{T},R_{T})\} \tag{6}\]  
where \(T\) denotes the maximum value of time (i.e., the time range of the change of the object's position over time).