SlowGuess/NeurIPS_2023_OCR huggingface dataset:
2) Dynamic Indoor Scene Dataset: We also collect another synthetic dynamic 3D dataset, which includes 4 indoor scenes with multiple complex 3D objects undergoing different rigid body motions. There are about 4 objects such as tables or chairs in each 3D scene. Basically, such an indoor dataset aims to simulate potential scenarios for robotics or VR applications to understand dynamic 3D scenes.
Since the indoor scene is significantly more challenging, for each 3D scene, we collect RGB images at 30 viewing angles over 1 second, where each viewing angle has 60 frames captured. Similarly, we reserve the first 45 frames at randomly picked 25 viewing angles as the training split, i.e., 1125 frames, while leaving the 45 frames at the remaining 5 viewing angles for testing interpolation ability, i.e., 225 frames, and keeping the last 15 frames at all 30 viewing angles for evaluating future frame extrapolation, i.e., 450 frames. The ground truth object segmentation masks are also collected for evaluating the semantic decomposition capability in Section 4.2. More details are in Appendix.
While existing dynamic 3D scene modeling techniques and the commonly used datasets in literature are mainly designed for novel view rendering/interpolation within the training time period, rather than for extrapolation beyond the training time period, we evaluate our method on two selected scenes from a real-world dataset: NVIDIA Dynamic Scene[73]. It captures real-world dynamic scenes by a static camera rig with 12 cameras. For each scene, we clip 60 frames with reasonable and predictable motions. We reserve the first 46 frames at randomly picked 11 cameras as the training split, i.e., 506 frames, while leaving the 46 frames at the remaining 1 camera for testing interpolation ability, i.e., 46 frames for novel view synthesis within the training time period, and keeping the last 14 frames at all 12 cameras for evaluating future frame extrapolation, i.e., 168 frames.
Baselines: We carefully choose three representative groups of methods as our baselines: 1) dense frame optimization method T-NeRF [46] and NSFF [33], 2) canonical frame optimization methods D-NeRF [46] and TiNeuVox [18], 3) PINN methods T-NeRF $_{PINN}$  and HexPlane $_{PINN}$ . Both methods are adapted by us via integrating a separate velocity field supervised by the same PINN losses as ours.
Metrics: For evaluating both interpolation and future frame extrapolation and motion transfer, the standard metrics PSNR, SSIM, and LPIPS scores are reported across testing views. For evaluating
Table 1: Quantitative results of all methods for both novel view interpolation and future frame extrapolation on Dynamic Object Dataset and Dynamic Indoor Scene Dataset.
Dynamic Object Dataset	Dynamic Indoor Scene Dataset
Interpolation	Extrapolation	Interpolation	Extrapolation
PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
T-NeRF[46]	13.163	0.709	0.353	13.818	0.739	0.324	24.944	0.742	0.336	22.242	0.700	0.363
D-NeRF[46]	14.158	0.697	0.352	14.660	0.737	0.312	25.380	0.766	0.300	20.791	0.692	0.349
TiNeuVox[18]	27.988	0.960	0.063	19.612	0.940	0.073	29.982	0.864	0.213	21.029	0.770	0.281
T-NeRFPINN	15.286	0.794	0.293	16.189	0.835	0.230	16.250	0.441	0.638	17.290	0.477	0.618
HexPlanePINN	27.042	0.958	0.057	21.419	0.946	0.067	25.215	0.763	0.389	23.091	0.742	0.401
NSFF[33]	-	-	-	-	-	-	29.365	0.829	0.278	24.163	0.795	0.289
NVFi(Ours)	29.027	0.970	0.039	27.594	0.972	0.036	30.675	0.877	0.211	29.745	0.876	0.204
Table 2: Quantitative results of our method and baselines on the NVIDIA Dynamic Scene dataset.
Truck	Skating
Interpolation	Extrapolation	Interpolation	Extrapolation
PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓	PSNR↑	SSIM↑	LPIPS↓
TiNeuVox[18]	27.230	0.846	0.229	24.887	0.848	0.209	29.377	0.889	0.202	24.224	0.878	0.220
HexPlanePINN	25.494	0.768	0.337	24.991	0.768	0.325	24.447	0.867	0.225	23.955	0.868	0.232
NVFi(Ours)	27.276	0.840	0.235	28.269	0.855	0.220	26.999	0.848	0.227	28.654	0.896	0.208
semantic decomposition, the Average Precision (AP), Panoptic Quality (PQ) and F1 scores with an IoU threshold of 0.5, together with the mean Intersection over Union (mIoU) scores are reported.
4.1 Evaluation of Future Frame Extrapolation
We first evaluate the extrapolation capability of our framework. In particular, our method and 5 baselines except NSFF are trained from scratch on each of the 6 objects in Dynamic Object dataset, and all methods on each of the 4 scenes in Dynamic Indoor Scene dataset, all in a scene-specific fashion. In total,  $(6\times 6) + (7\times 4) = 64$  models are trained for comparison. The keyframe number  $K$  is set as 16 in our method for Dynamic Object Dataset and 4 for Dynamic Indoor Scene Dataset. As for real-world NVIDIA Dynamic Scene dataset, we evaluate our model and 2 baselines with comparable performance on our own datasets, where the keyframe number  $K$  is set as 4 in our method.
Analysis: Table 1 compares all methods regarding the quality of view synthesis for future frame extrapolation. The view synthesis for interpolation is also included for comparison. It can be seen that: 1) our NVFi achieves significantly better results than all baselines on both dynamic datasets, particularly on the critical task of future frame extrapolation, although the strong baseline TiNeuVox shows excellent performance for interpolation. 2) Naively adding physics priors into an existing dynamic NeRF tends to be inferior as shown by T-NeRF $_{PINN}$  and HexPlane $_{PINN}$ . This clearly verifies the effectiveness of our special design of the joint keyframe and interframe optimization strategy. More qualitative results are in Figure 4(a). Table 2 compares our method and two best baselines. It can be seen that, even if our NVFi only gets comparable performance in interpolation, it still achieves the best performance in extrapolation without any performance drop compared with interpolation thanks to the accurate motion predictions. More qualitative results are in Figure 3.
4.2 Evaluation of 3D Semantic Scene Decomposition
Having the well-trained keyframe dynamic radiance field  $f_{\Theta}$  and interframe velocity field  $g_{\Phi}$  for each 3D scene in the Dynamic Indoor Scene dataset in Section 4.1, naturally, all individual 3D objects such as chairs and tables undergoing different movements are supposed to be automatically discovered, segmented, and tracked without needing any extra object annotations as supervision signals.
In order to achieve this desirable unsupervised object decomposition objective, a naive strategy is to query the velocity values of dense 3D points in space, followed by a velocity clustering module to group points into objects. However, such a strategy fundamentally fails to recognize object shapes but just identifies similar motions, thereby tracking objects is also infeasible. A more elegant strategy is to directly learn an object code  $\pmb{o}_p$  (usually one-hot) for each 3D point  $\pmb{p}$  within the entire 3D scene volume at the initial timestamp  $t = 0$ , without retraining any neural layer of the well-trained networks  $f_{\Theta}$  and  $g_{\Phi}$  but just using them. This would clearly allow all dynamic 3D objects in the scene to be segmented and tracked over time. To this end, we simply introduce a simple 4-layer MLP as the
semantic scene decomposition network  $h_{\Psi}$ . It takes a 3D point  $\pmb{p}$  as input, directly regressing its object code  $\pmb{o}_{p}$ , i.e.,  $\pmb{o}_{p} = h_{\Psi}(\pmb{p})$  which is optimized using the following steps:
- First, given the well-trained keyframe dynamic radiance field  $f_{\Theta}$ , we uniformly sample dense 3D points at timestamp  $t = 0$ , obtaining their density values. Only 3D points with sufficiently large densities are kept as valid points for the subsequent steps.
- Second, the valid 3D points are fed into our new object segmentation network  $h_{\Psi}$  (initialized but yet to be trained), obtaining their corresponding object codes.
- Third, the valid points are transported to their correspondences at a random timestamp  $t'$  using the well-trained velocity field  $g_{\Phi}$ . Motion vectors of these points from 0 to  $t'$  are computed.
- Lastly, given per-point motions, we employ the dynamic rigid consistency and spatial smoothness losses proposed in OGC [54] to optimize the object segmentation network  $h_{\Psi}$ .
Once the object segmentation network  $h_\Psi$  is learned, all dynamic objects at time  $t = 0$  are segmented. With the aid of the well-trained velocity field, in any subsequent timestamps, all these identified objects can be naturally tracked. In addition, with the aid of well-trained keyframe dynamic radiance field  $f_\Theta$ , we simply use the accumulated weights computed in volume rendering to combine point object codes along a given light ray, thus rendering accurate object segmentation 2D masks for any camera poses at any given timestamps. More details of the implementation are in Appendix.
We evaluate the semantic scene decomposition ability on the Dynamic Indoor Scene dataset. In particular, we render all 2D object segmentation masks from our network at the 30 viewing angles over 60 frames for all scenes, i.e., 7200 2D masks, and then evaluate them against ground truth masks. For a fair comparison, we also train a similar object segmentation network for the
Table 3: Quantitative results of scene decomposition on the Synthetic Indoor Scene dataset.
AP↑	PQ↑	F1↑	mIoU↑
Mask2Former [11]	65.37	73.14	78.29	64.42
D-NeRF [46]	57.26	46.15	59.02	46.58
NVFi(Ours)	91.21	78.74	93.75	67.64
baseline D-NeRF [46] at time  $t = 0$ , using its learned deformation vectors as supervision signals and tracking signals. Note that, the deformation vectors are converted back as motion vectors. In addition, we include an image-based object segmentation method, the powerful Mask2Former [11] pre-trained on COCO [34] dataset, as a fully-supervised baseline. More implementation details are in Appendix.
Analysis: As shown in Table 3, we can see that: 1) Our object segmentation performance is superior to D-NeRF [46], essentially because our velocity field learns better scene dynamics than the deformation field, thus enabling the object segmentation network to be better optimized. 2) We also clearly surpass the powerful pre-trained Mask2Former [11] on all metrics. The reasons are two-fold. First, we fundamentally rely on motion patterns rather than appearances to discover objects, thus being able to generalize to unseen object types ("Genome") or scenes ("Factory") better than Mask2Former [11]. Second, our learned object field inherently leverages multi-view consistency, thus allowing the segmentation of partially occluded objects. Figure 4(b) shows qualitative results.
4.3 Evaluation of Motion Transfer
We further demonstrate the ability of our model to transfer a well-trained velocity field to another separately trained static scene. All objects in the new scene are expected to undergo the same dynamics as learned in the velocity field. The more accurate the learned velocity field, the more realistic and entertaining the new 3D scene will be, after applying the learned dynamics.
In order to evaluate the performance, we create a new 3D scene, called Gnome-new, being similar to the scene Gnome in our Dynamic Indoor Scene dataset. We apply the same dynamics of Gnome on Gnome-new, recording  $30^{*}60 = 1800$  frames as its ground truth observations. To explicitly show the advantage of our learned disentangled velocity field, we sepa
Table 4: Quantitative results of motion transfer on Synthetic Indoor Scene dataset.
PSNR↑	SSIM↑	LPIPS↓
D-NeRF [46]	16.124	0.327	0.550
NVFi(Ours)	16.178	0.334	0.551
rately train a static TensoRF [7] model for Gnome-new only using its frames at time  $t = 0$ . Note that, any other NeRF variants are also applicable here. We then pick up the well-trained velocity field of Gnome in Section 4.1, after which we directly apply the learned velocity field on the newly trained static TensoRF model, rendering  $30^{*}60$  frames for a comparison with the ground truth images. Similarly, we apply the deformation field learned by D-NeRF in Section 4.1 in the same transferring pipeline, rendering 2D images for comparison. More implementation details are in Appendix.
Figure 3: Qualitative results of baselines and our method on NVIDIA Dynamic Scene dataset.
Analysis: Figure 4(c) shows that our method clearly keeps the geometry and appearance of the new static object, thanks to the accurately learned disentangled velocity field, whereas D-NeRF [46] fails to do so. However, we observe that in Table 4 our advantage is not so significant. This is because the static reconstruction of Gnome-new lacks supervision signals for ground planes occluded by objects, leading to rendering artifacts in these regions when objects are moved away by motion transfer. Our strong ability of motion transfer is further validated by additional experiments on our Dynamic Object Dataset in Appendix.
(1) Without Joint Optimization: We only use  $\ell_{keyframe}$  to train the keyframe dynamic radiance field  $f_{\Theta}$ , followed by the  $(\ell_{divergence\_free} + \ell_{momentum} + \ell_{interframe})$  to separately train the velocity field  $g_{\Phi}$ .
(2) Removing Physics Constraints: The PINN losses  $(\ell_{divergence\_free} + \ell_{momentum})$  are removed to train  $g_{\Phi}$ .
(3) Choice of Keyframe Number  $K$ : We set the keyframe number  $K$  as 8 and 32, while we choose  $K = 16$  in main experiments.
(4) Reducing the Number of Cameras: we reduce the number of cameras used in our Dynamic Object datasets to half of the number, i.e., 6 cameras, and one quarter of the number, i.e., 3 cameras.
Table 5 and Table 6 shows the ablation results for future frame extrapolation on our Dynamic Object dataset. We can see that: 1) The joint keyframe and interframe
optimization strategy is critical to enable our method to learn accurate velocity field as well as dynamic radiance field. 2) Once the keyframe number  $K$  becomes larger, the extrapolation capability clearly drops, validating that the dense supervision is inferior to help learn physics velocity overall. 3) The extremely sparse camera views are unlikely to capture sufficient visual information for physical motion learning. More ablation results are in Appendix.
Table 5: Quantitative results of ablation studies on Dynamic Object dataset.
Joint	Physics	#K	Extrapolation
PSNR↑	SSIM↑	LPIPS↓
✓	✓	16	27.594	0.972	0.036
X	✓	16	24.792	0.955	0.059
✓	X	16	25.537	0.968	0.040
✓	✓	8	27.490	0.974	0.036
✓	✓	32	24.902	0.964	0.037
Table 6: Quantitative results of ablation studies on Dynamic Object dataset.
Camera Numbers	Extrapolation
PSNR↑	SSIM↑	LPIPS↓
12	27.594	0.972	0.036
6	25.114	0.959	0.122
3	21.370	0.917	0.084
In this paper, we extend the appealing radiance field to represent dynamic 3D scenes from multiview videos. Unlike most of the existing methods which focus on novel view synthesis within the training time period, our method learns to disentangle the physical velocity field from the geometry and appearance of 3D scenes by jointly optimizing two neural networks: the keyframe dynamic radiance field and the interframe velocity field. Extensive experiments on three dynamic datasets demonstrate that our framework learns accurate velocity, enabling successful applications in future frame extrapolation, semantic scene decomposition, and motion transfer.
(a) Future Frame Extrapolation
Figure 4: Qualitative results of baselines and our method on the three tasks. More qualitative results can be found in Appendix and our project page: https://vlar-group.github.io/NVFi.html
Acknowledgements: This work was supported in part by Research Grants Council of Hong Kong under Grants 25207822 & 15225522.
[1] B. Attal, E. Laidlaw, A. Gokaslan, C. Kim, C. Richardt, J. Tompkin, and M. O'Toole. ToRF: Time-of-Flight Radiance Fields for Dynamic Scene View Synthesis. NeurIPS, 2021.
[2] D. Baieri, S. Esposito, F. Maggioli, and E. Rodola. Fluid Dynamics Network: Topology-Agnostic 4D Reconstruction via Fluid Dynamics Priors. arXiv:2303.09871, 2023.
[3] J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, and R. M.-b. Pratul. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. ICCV, 2021.
[4] J. T. Barron, K. Park, S. M. Seitz, and R. Martin-brualla. Nerfies: Deformable Neural Radiance Fields. ICCV, 2021.
[5] H. Cai, W. Feng, X. Feng, Y. Wang, and J. Zhang. Neural Surface Reconstruction of Dynamic Scenes with Monocular RGB-D Camera. NeurIPS, 2022.
[6] A. Cao and J. Johnson. HexPlane: A Fast Representation for Dynamic Scenes. CVPR, 2023.
[7] A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su. TensoRF: Tensorial Radiance Fields. ECCV, 2022.
[8] H. Chen, R. Wu, E. Grinspun, C. Zheng, and P. Y. Chen. Implicit Neural Spatial Representations for Time-dependent PDEs. arXiv:2210.00124, 2022.
[9] R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud. Neural Ordinary Differential Equations. NeurIPS, 2018.
[10] Z. Chen and H. Zhang. Learning Implicit Fields for Generative Shape Modeling. CVPR, 2019.
[11] B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar. Masked-attention Mask Transformer for Universal Image Segmentation. CVPR, 2022.
[12] J. Chibane, A. Mir, and G. Pons-Moll. Neural Unsigned Distance Fields for Implicit Function Learning. NeurIPS, 2020.
[13] C. B. Choy, D. Xu, J. Gwak, K. Chen, and S. Savarese. 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction. ECCV, 2016.
[14] M. Chu, L. Liu, Q. Zheng, E. Franz, H. P. Seidel, C. Theobalt, and R. Zayer. Physics informed neural fields for smoke reconstruction with sparse data. TOG, 2022.
[15] Y. Deng, H.-X. Yu, J. Wu, and B. Zhu. Learning Vortex Dynamics for Fluid Inference and Prediction. ICLR, 2023.
[16] Y. Du, Y. Zhang, and J. B. Tenenbaum. Neural Radiance Flow for 4D View Synthesis and Video Processing. ICCV, 2021.
[17] H. Fan, H. Su, and L. Guibas. A Point Set Generation Network for 3D Object Reconstruction from a Single Image. CVPR, 2017.
[18] J. Fang, X. Wang, and M. Nießner. Fast Dynamic Radiance Fields with Time-Aware Neural Voxels. SIGGRAPH Asia, 2022.
[19] S. Fridovich-Keil, G. Meanti, F. Warburg, B. Recht, and A. Kanazawa. K-Planes: Explicit Radiance Fields in Space, Time, and Appearance. CVPR, 2023.
[20] G. Gafni, J. Thies, M. Zollhöfer, and M. Nießner. Dynamic neural radiance fields for monocular 4D facial avatar reconstruction. CVPR, 2021.
[21] C. Gao, A. Saraf, J. Kopf, and J.-B. Huang. Dynamic View Synthesis from Dynamic Monocular Video. ICCV, 2021.
[65] S. Wang, H. Wang, and P. Perdikaris. Learning the solution operator of parametric partial differential equations with physics-informed DeepONets. Science Advances, 7(40), 2021.
[66] T.-C. Wang, A. Mallya, and M.-Y. Liu. One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing. CVPR, 2021.
[67] Z. Wang, T. Bagautdinov, S. Lombardi, T. Simon, J. Saragih, J. Hodgins, and M. Zollhöfer. Learning Compositional Radiance Fields of Dynamic Human Heads. CVPR, 2021.
[68] C.-Y. Weng, B. Curless, P. P. Srinivasan, J. T. Barron, and I. Kemelmacher-Shlizerman. Human-NeRF: Free-viewpoint Rendering of Moving People from Monocular Video. CVPR, 2022.
[69] W. Xian, J.-B. Huang, J. Kopf, and C. Kim. Space-time Neural Irradiance Fields for Free-Viewpoint Video. CVPR, 2021.
[70] H. Xu, T. Alldieck, and C. Sminchisescu. H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion. NeurIPS, 2021.
[71] B. Yang, S. Rosa, A. Markham, N. Trigoni, and H. Wen. Dense 3D Object Reconstruction from a Single Depth View. TPAMI, 2019.
[72] B. Yang, S. Wang, A. Markham, and N. Trigoni. Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction. IJCV, 128:53-73, 2020.
[73] J. S. Yoon, K. Kim, O. Gallo, H. S. Park, and J. Kautz. Novel view synthesis of dynamic scenes with globally coherent depths from a monocular camera, 2020.
[74] M. You and J. Hou. Decoupling Dynamic Monocular Videos for Dynamic View Synthesis. arXiv:2304.01716, 2023.
[75] W. Yuan, Z. Lv, T. Schmidt, and S. Lovegrove. STaR: Self-supervised Tracking and Reconstruction of Rigid Objects in Motion with Neural Rendering. CVPR, 2021.
[76] S. Zhi, T. Laidlow, S. Leutenegger, and A. J. Davison. In-Place Scene Labelling and Understanding with Implicit Scene Representation. ICCV, 2021.
[77] C. Zou, E. Yumer, J. Yang, D. Ceylan, and D. Hoiem. 3D-PRNN: Generating Shape Primitives with Recurrent Neural Networks. ICCV, 2017.
The appendix includes:
- Rendering equation.
- Implementation details of keyframe dynamic radiance field and velocity field.
Additional details of physics prior.
- Implementation details of joint optimization.
Additional details of datasets.
- Implementation details about semantic scene decomposition.
- Implementation details about motion transfer.
Additional quantitative results for ablation study.
- Additional quantitative & qualitative results for future frame extrapolation.
- Additional quantitative & qualitative results for semantic scene decomposition.
Additional qualitative results for motion transfer.
In addition, we provide a demo video for better visualization of qualitative results on multiple tasks, appended in the supplementary materials.
A.1 Rendering Equation
Neural radiance fields were initially introduced [37] to model 3D scenes by associating the coordinate  $(x,y,z)$  and view direction  $(\theta, \phi)$  of each point in space with its color, represented by vector  $\mathbf{c}$ , and density, denoted by  $\sigma$ . We keep the same rendering equation to obtain the expected color  $C(\mathbf{r}_i, t_i)$  of a pixel in the image captured by a camera at time  $t_i$ , a ray  $\mathbf{r}_i(s) = \mathbf{o}_i + s\mathbf{d}_i$  is involved, which marches from the camera's center towards the pixel. Here,  $\mathbf{o}_i$  and  $\mathbf{d}_i$  represent the ray's origin and direction, respectively, while  $s$  signifies the distance from a point to the camera, ranging from a predefined near bound  $s_n$  to a far bound  $s_f$ . The pixel color is rendered by sampling a series of points along the ray and performing classical volume rendering:
$$
\hat {C} (\mathbf {r}) = \sum_ {i = 1} ^ {N} T _ {i} (1 - \exp (- \sigma_ {i} \delta_ {i})) \mathbf {c} _ {i}, \tag {11}
$$
$$
T _ {i} = \exp \left(- \sum_ {j = 1} ^ {i - 1} \sigma_ {j} \delta_ {j}\right),
$$
where  $\delta_{i}$  represents the distance between the  $i_{\mathrm{th}}$  and  $(i + 1)_{\mathrm{th}}$  sample point, and  $\mathbf{N}$  denotes the number of sampled points. Equation (11) connects real 3D points with image pixels by accumulating colors  $\mathbf{c}_i$  and densities  $\sigma_{i}$  of sample points along the ray.
A.2 Implementation Details of Keyframe Dynamic Radiance Field
We adapt HexPlane[6] to parameterize volumetric keyframes. It is a direct extension of static TensoRF[7] to a dynamic radiance field. Instead of using concatenation to combine the features from different planes as TensoRF and HexPlane, we use Hadamard Production as used in K-Plane[19], which shows stronger ability in representing complex geometries. The formal definition of our Keyframe Dynamic Radiance Field is as follows:
If we denote  $\tau_{k}$  as the time step corresponding to the  $k^{\mathrm{th}}$  keyframe, we can write:
$$
\mathbf {A} \left(\mathbf {x} _ {i}, \tau_ {k}\right) = \mathbf {B} \left(\mathbf {f} _ {1} \left(x _ {i}, y _ {i}\right) \odot \mathbf {g} _ {1} \left(z _ {i}, \tau_ {k}\right) \odot \mathbf {f} _ {2} \left(x _ {i}, z _ {i}\right) \odot \mathbf {g} _ {2} \left(y _ {i}, \tau_ {k}\right) \odot \mathbf {f} _ {3} \left(y _ {i}, z _ {i}\right) \odot \mathbf {g} _ {3} \left(x _ {i}, \tau_ {k}\right)\right), \tag {12}
$$
$$
\boldsymbol {\sigma} \left(\mathbf {x} _ {i}, \tau_ {k}\right) = \mathbf {1} ^ {\top} \left(\mathbf {h} _ {1} \left(x _ {i}, y _ {i}\right) \odot \mathbf {k} _ {1} \left(z _ {i}, \tau_ {k}\right) \odot \mathbf {h} _ {2} \left(x _ {i}, z _ {i}\right) \odot \mathbf {k} _ {2} \left(y _ {i}, \tau_ {k}\right) \odot \mathbf {h} _ {3} \left(y _ {i}, z _ {i}\right) \odot \mathbf {k} _ {3} \left(x _ {i}, \tau_ {k}\right)\right),
$$