SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Algorithm 1 Conditional IMLE Training Procedure
Require: The set of inputs  $\{\mathbf{x}_i\}_{i = 1}^n$  and the set of corresponding observed outputs  $\{\mathbf{y}_i\}_{i = 1}^n$  Initialize the parameters  $\theta$  of the generator  $T_{\theta}$    
for  $p = 1$  to  $N_{outer}$  do Pick a random batch  $S\subseteq \{1,\dots ,n\}$    
for  $i\in S$  do Randomly generate i.i.d.  $m$  latent codes  $\mathbf{z}_1,\ldots ,\mathbf{z}_m$ $\tilde{\mathbf{y}}_{i,j}\gets T_{\theta}(\mathbf{x}_i,\mathbf{z}_j)\forall j\in [m]$ $\sigma (i)\gets \arg \min_{j}d(\mathbf{y}_{i},\tilde{\mathbf{y}}_{i,j})\forall j\in [m]$    
end for   
for  $q = 1$  to  $N_{inner}$  do Pick a random mini-batch  $\tilde{S}\subseteq S$ $\theta \leftarrow \theta -\eta \nabla_{\theta}\left(\sum_{i\in \tilde{S}}d(\mathbf{y}_i,\tilde{\mathbf{y}}_{i,\sigma (i)})\right) / |\widetilde{S} |$    
end for   
end for   
return  $\theta$
In our specific context,  $\mathbf{x}_i$  is the feature map,  $T_{\theta}$  is the modified point feature renderer,  $\mathbf{y}_i$  is the target RGB image. We first load pre-trained weights for all model parameters except the additional MLP for the latent variable before training. During test time, we randomly sample latent codes to change the exposure of the rendered image.
B.1 Qualitative Comparison
We include extra qualitative results for NeRF Synthetic dataset [22] in Figure 14. Furthermore, we include qualitative comparisons between our method and the more competitive baselines, based on metric scores, on the Tanks & Temples subset in Figure 15.
Figure 14: Qualitative comparison of novel view synthesis on the NeRF Synthetic dataset [22].
Figure 15: Qualitative comparison of novel view synthesis between our method, PAPR, and the more competitive baselines on Tanks & Temples [13] subset.
B.2 Point Cloud Learning
We show the point clouds generated by our method for each scene, depicted in Figure 16 for the NeRF Synthetic dataset [22], and Figure 17 for the Tanks & Temples [13] subset. These results demonstrate the effectiveness of our method in learning high-quality point clouds that correctly capture the intricate surfaces of the scenes.
Figure 16: Point clouds learnt by our method on the NeRF Synthetic dataset [22].
Figure 17: Point clouds learnt by our method on the Tanks & Temples [13] subset.
B.3 Zero-shot Geometry Editing
We present comparisons with point-based baselines on two scenes in which we apply non-volume preserving stretching transformations. In the first scene, we stretch the back of the chair, and in the second case, we stretch the tip of the microphone. As shown in Figure 18, NPLF [24], DPBRF [53], Point-NeRF [48] and Gaussian Splitting [11] either create holes or produce significant noise after the transformation. Additionally, SNP [56] fails to preserve the texture details following the edits, such as the golden embroidery pattern on the back of the chair and the mesh grid pattern on the tip of the microphone. In contrast, our method successfully avoids creating holes and effectively preserves the texture details after the transformation.
Figure 18: Qualitative comparisons among our method, PAPR, and point-based baselines following non-volume preserving stretching transformations. The rendered images before and after the transformations are shown. The results after the transformations are rendered by manipulating the point positions only during test time.
B.4 Quantitative Results
We provide the metric scores broken down by scene on both datasets. Table 2 shows the per-scene scores for the NeRF Synthetic dataset [22] and Table 3 shows the scores for the Tanks & Temples [13] subset.
Chair	Drums	Lego	Mic	Materials	Ship	Hotdog	Ficus	Avg.
PSNR↑
NPLF [24]	19.69	16.45	20.70	19.84	16.03	16.04	18.56	19.55	18.36
DPBRF [53]	26.51	19.38	25.09	29.26	26.20	21.93	31.87	24.61	25.61
SNP [56]	28.81	21.74	25.75	28.17	24.04	23.55	31.74	24.23	26.00
Point-NeRF [48]	30.24	23.60	23.42	31.75	24.77	18.72	26.56	28.40	25.93
Gaussian Splatting [11]	31.40	22.44	28.18	31.96	27.39	19.84	36.19	24.66	27.76
PAPR(Ours)	33.59	25.35	32.62	35.64	29.54	26.92	36.40	36.50	32.07
SSIM↑
NPLF [24]	0.798	0.737	0.763	0.886	0.739	0.678	0.792	0.842	0.780
DPBRF [53]	0.929	0.834	0.885	0.951	0.895	0.713	0.953	0.909	0.884
SNP [56]	0.938	0.889	0.907	0.964	0.909	0.823	0.961	0.917	0.914
Point-NeRF [48]	0.971	0.927	0.904	0.985	0.929	0.761	0.934	0.973	0.923
Gaussian Splatting [11]	0.957	0.901	0.935	0.982	0.938	0.792	0.978	0.953	0.929
PAPR(Ours)	0.986	0.951	0.981	0.993	0.972	0.904	0.988	0.994	0.971
LPIPSVgg↓
NPLF [24]	0.183	0.214	0.236	0.136	0.226	0.339	0.226	0.143	0.213
DPBRF [53]	0.098	0.174	0.142	0.070	0.143	0.288	0.093	0.095	0.138
SNP [56]	0.049	0.081	0.057	0.025	0.072	0.167	0.036	0.050	0.110
Point-NeRF [48]	0.065	0.125	0.152	0.038	0.148	0.298	0.143	0.067	0.129
Gaussian Splatting [11]	0.049	0.108	0.077	0.019	0.063	0.273	0.037	0.049	0.084
PAPR(Ours)	0.018	0.055	0.027	0.007	0.036	0.129	0.021	0.010	0.038
Table 2: Comparison of image quality metrics (PSNR, SSIM and LPIPS [54]), broken down by scene, for the NeRF Synthetic dataset [22].
We provide additional quantitative results for the ablation study. Figure 19 shows the PSNR and SSIM scores for the methods using different number of points. Additionally, Table 4 shows the PSNR and SSIM scores for the various choices of the ray-dependent point embedding. These results validate the effectiveness of our proposed method.
Figure 19: Extra image quality metrics (PSNR, SSIM) for the ablation study on different number of points. Higher values of PSNR and SSIM scores are better.
Ignatius	Truck	Barn	Caterpillar	Family	Avg.
PSNR↑
NPLF [24]	24.60	20.01	20.24	17.69	23.39	21.19
DPBRF [53]	18.22	16.20	14.68	17.23	19.93	17.25
SNP [56]	28.41	24.32	25.27	22.62	31.31	26.39
Point-NeRF [48]	28.59	25.57	20.97	17.71	30.89	24.75
Gaussian Splatting [11]	26.48	22.32	26.22	23.27	35.79	26.81
PAPR(Ours)	28.40	26.98	27.06	26.79	34.39	28.72
SSIM↑
NPLF [24]	0.882	0.725	0.634	0.719	0.846	0.761
DPBRF [53]	0.797	0.563	0.422	0.614	0.776	0.634
SNP [56]	0.944	0.876	0.832	0.858	0.958	0.894
Point-NeRF [48]	0.960	0.926	0.799	0.797	0.970	0.890
Gaussian Splatting [11]	0.937	0.867	0.860	0.889	0.983	0.907
PAPR(Ours)	0.956	0.931	0.896	0.932	0.983	0.940
LPIPSVgg↓
NPLF [24]	0.120	0.265	0.381	0.275	0.157	0.240
DPBRF [53]	0.174	0.317	0.484	0.334	0.194	0.301
SNP [56]	0.077	0.177	0.278	0.196	0.071	0.160
Point-NeRF [48]	0.085	0.162	0.355	0.237	0.084	0.184
Gaussian Splatting [11]	0.095	0.186	0.231	0.155	0.032	0.140
PAPR(Ours)	0.072	0.108	0.157	0.118	0.031	0.097
Table 3: Comparison of image quality metrics (PSNR, SSIM and LPIPS [54]), broken down by scene, for the Tanks & Temples [13] subset.
Lego	Mic
PSNR ↑	SSIM ↑	PSNR ↑	SSIM ↑
Full Model	32.62	0.981	35.64	0.993
w/o pi	31.67	0.976	32.00	0.985
w/o pi, ti,j	13.26	0.701	18.73	0.902
Table 4: Extra image quality metrics (PSNR, SSIM) for the ablation study on different designs for the ray-dependent point embedding. Higher values of PSNR and SSIM scores are better.