SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Choosing the appropriate basis function for an application requires consideration of two fundamental properties of basis functions: their linear independence and learning capacity. Linear independence is crucial for satisfying hard constraints, whereas learning capacity is key for fitting the inductive bias. In this ablation study, we train various neural basis functions discussed in Sec. 3.2 to regularize the condition number of the matrix  $\mathbf{A}_f$ , which is used to constrain a 2D function over 4096 points. A large condition number often corresponds to a substantial error in satisfying hard constraints. This study offers insights into how each design is prone to the ill-conditioning of the linear system. For a fair comparison, we use the same architecture for all neural encoders—a 2-layer MLP with Tanh activations and a total of 1.3M trainable parameters.
As shown in Fig. 5, the Gaussian kernel basis is well-conditioned throughout training, while all other methods struggle to reduce the condition number.
In Sec. B.2, we also discovered that the Gaussian kernel basis, when constrained, exhibits the superior learning capacity for fitting the BRDF. In contrast, the performance of other methods was found to be extremely poor. Note that the dot-product kernel and the constrained layer are the basis functions used in [41] and [24], respectively. Based on our findings, we strongly recommend employing the Gaussian kernel basis function when performing constrained optimization for neural fields.
Figure 5: Condition number of different neural basis functions in the following order (left to right): constraint layer (pink), hypernet basis (yellow), dot-product kernel (blue), and Gaussian kernel (black) throughout training.
B Learning Material Appearance
B.1 Experiment Details
In the experiments on material appearance fitting, all neural networks have a single hidden layer and we controlled the network width so that all methods have roughly 200k parameters for a fair comparison. Both baseline FFN and kernel FFN use a frequency of 16, meaning that the inputs are mapped to encodings of size 38 before feeding to the MLP. The kernel FFN model uses FFN as an encoder which has a width of 248 and outputs a latent vector of dim 512. The baseline FFN MLP has a width of 718. The kernel SIREN model has a width of 256 and also outputs a latent vector of size 512. The baseline SIREN and NBRDF have a width of 442. We use Adam optimizer with a learning rate of  $5 \times 10^{-4}$  for all methods except for SIREN baseline and kernel SIREN, which use a learning rate of  $1 \times 10^{-4}$  for more stable performance.
For constraint points, we sample the angles  $\theta_h$ ,  $\theta_d$  and  $\phi_d$  as in Rusinkiewicz's parameterization [33]. A figure of those angles is shown in Fig. 6. We sample  $\theta_d$  and  $\phi_d$  uniformly at random within range  $[0, \pi/2]$  and  $[0, 2\pi]$  respectively. Half of the  $\theta_h$  are also sampled uniformly at random within  $[0, \pi/2]$ , whereas the other half are sampled from a Gaussian distribution with a mean of 0 and a standard deviation of 0.1. Those angles are then converted to 6D in-going and out-going directions as inputs to the networks.
B.2 Ablation Studies
In Tab. 4 and Fig. 7, we report the ablation results of fitting the BRDF with constraints, including the dot-product kernel using SIREN as the encoder, the hypernet basis with SIREN as the main network, and the constrained layer integrated into SIREN. The hypernet basis uses a SIREN that has a width of 442 as the main network, and a ReLU MLP with one hidden layer with 100 nodes as the hypernetwork. The constraint layer has a width of 319. The hypernet basis and constraint layer produce extremely poor results and are completely unable to perform this task. While the dot-product kernel performs better than both the hypernet basis and the constraint layer, it still significantly underperforms when compared to the Gaussian kernel.
B.3 Learning Curves
In Fig. 8 and Tab. 5, we report the learning curves, training times, and inference times for various approaches in the BRDF reconstruction experiment. CNF with a Gaussian kernel basis outperforms other approaches in terms of convergence, while maintaining similar training and inference efficiency.
Figure 6: Angles for sampling constraint points for material learning. Figure from [33].
Table 4: Ablations on material appearance learning task. All methods here are based on SIREN [34].
RMSE×10-2	PSNR	SSIM
Hypernet Basis	39.02 ± 16.92	9.74 ± 6.41	0.70 ± 0.06
Dot-product Kernel	9.49 ± 11.32	28.33 ± 13.22	0.86 ± 0.13
Constraint Layer	41.85 ± 14.14	8.37 ± 4.37	0.70 ± 0.06
Gaussian Kernel	0.60 ± 0.54	47.95 ± 7.99	0.99 ± 0.01
Figure 7: Ablation on the material appearance fitting.
Figure 8: Learning curves of various approaches for BRDF reconstruction.
Table 5: Training and inference time (in seconds) of various approaches for BRBF reconstruction.
Method	Training	Inference
SIREN [34]	578.89	0.3348
NBRDF [35]	19.702	0.2966
Constraint layer [26]	577.49	0.2539
Gaussian kernel	1101.3	0.7284
Hypernet basis	2997.5	0.6933
Hypernet kernel	1784.1	1.1692
B.4 Full Results
The complete rendering results for all materials included in the MERL dataset [21] are shown in Fig. 9, Fig. 10, Fig. 11, Fig. 12, Fig. 13, and Fig. 14, and are available on the project page https://zfc946.github.io/CNF.github.io/. We also show the SSIM errors along with the renderings.
C Interpolatory Shape Reconstruction
C.1 Experiment Details
C.1.1 Exact Normal Interpolation
In the experiments on exact normal interpolation, the shapes are represented by the level sets of our hypernet kernel. The encoder of our hypernet is constructed of an MLP, which has a single hidden layers containing 1200 hidden units. Between each layer we use a softmax activation, with  $\beta = 10$  and the encoder maps to a feature space of size 800. We use a Gaussian kernel which is initialized with a standard deviation of 50. For all experiments shown, we use a maximum of 50 epochs, a learning rate of  $10^{-5}$ , and the Adam optimizer. To extract the zero-order level set of the implicit representation we use the Marching Cubes algorithm, on a uniform grid sampled at a step size of 1/60. For all shapes except the circle, we pretrain the weights of the hypernet encoder to first represent a circle with uniformly sampled constraints equal to the number of constraints of the target shape. The Eikonal loss function defined in Eq. 13 is defined on the domain  $\Omega = [-2,2] \times [-2,2]$ , and at each iteration we use 1000 points randomly sampled from  $\Omega$ .
C.1.2 Large Scale Constraints
In the demonstration of our patched-based approach, we reconstruct point clouds with 10,000 uniformly sampled points. Our current implementation of the patch-based solver doesn't support the hypernet kernel, so we regularize the reconstruction problem using a finite difference scheme [39, 42] and impose the point constraints of  $\Phi (\pmb {x}_i + \varepsilon \pmb {n}_i) = \varepsilon$  and  $\Phi (\pmb {x}_i - \varepsilon \pmb {n}_i) = -\varepsilon$ , where we set  $\varepsilon = 0.01$  for all reconstructions. As well as the tunable support size, our implementation also places a hard limit on the total number of constraints that are considered for each query point; for all experiments we limit the nearest neighborhood size to 60. Furthermore, as our current implementation does not support tensor batching, the run-time for the patch-based approach, with a marching cubes grid resolution of  $[160\times 160\times 160]$ , is approximately 8 hours.
In the context of sparse solvers, we acknowledge that our current implementation is not completely vectorized for fast GPU execution with batched training points, although the operation of the solver itself is highly efficient (execution within seconds). This limitation primarily arises due to the limited support of sparse matrix operations in popular ML frameworks. However, it is crucial to emphasize that this constraint does not reflect a shortcoming in CNF's design. We hope to attract more attention from the ML community to enhance support for sparse matrix operations within the frameworks to resolve this implementation constraint.
C.2 Evaluation of Normal Interpolation
To evaluate the merit of our normal interpolation approach for shape representation, we compare shape representation using our exact gradient constraints and pseudo-normal constraints as used in NKF [41] and SIREN [34]. We use the same hypernet kernel approach for all experiments, with the same training setup as Sec. C.1.1, except that we do not pretrain the hypernet encoder to represent a circle. For the pseudo-normal constraints, we set the value of  $\epsilon = 0.1$ .
In Fig. 15 we show a visual comparison between the reconstructions achieved when using pseudonormal constraints and exact gradient constraints. To quantitatively assess each approach we also report the mean normal error:
$$
\mathcal {E} _ {\boldsymbol {n}} = \frac {1}{N} \sum_ {i = 1} ^ {N} \| \boldsymbol {n} \left(\boldsymbol {x} _ {i}\right) - \hat {\boldsymbol {n}} \left(\boldsymbol {x} _ {i}\right) \|, \tag {21}
$$
where  $\pmb{n}(\pmb{x}_i) = \nabla_{\pmb{x}}\Psi (\pmb{x}_i)$  and  $\hat{\pmb{n}} (\pmb{x}_i)$  is the ground truth normal value at  $\pmb{x}_i$ . The results are tabulated in Table 6.
As is evident both from our qualitative visualization and quantitative evaluation, CNF is able to impose exact normal constraints on neural implicit surfaces during initialization and throughout training (to within floating point accuracy).
SIREN (*)	Pseudo-normal (†)	Pseudo-normal (*)	Exact normal (†)	Exact normal (*)
Circle	1.71 × 10-2	0.776	0.499	4.516 × 10-6	1.083 × 10-6
Line	1.02 × 10-3	3.817	1.589 × 10-2	2.737 × 10-6	6.963 × 10-6
Triangle	3.21 × 10-1	0.466	0.238	7.371 × 10-5	6.292 × 10-6
Diamond	1.05 × 10-1	1.588	0.277	1.450 × 10-5	3.368 × 10-6
Table 6: Evaluation of error in the normal vector of the implicit field, as defined in Eq. 21. The symbols  $\dagger$  and  $*$  refer to measurements taken on initialization and after training the field, respectively.
Figure 15: We assess the effectiveness of our precise normal interpolation technique for shape representation. We present the normal vector of the encoded implicit surface (red arrow) before and after training to minimize an Eikonal constraint, for fields bound by pseudo-normal constraints and our exact constraint method. We compare with the ground-truth shape normal vectors (black arrow); our approach produces a surface that faithfully represents both the constraint points and the constraint normal vector.
C.3 Learning Curves
In Fig. 16 and Tab. 7, we report the learning curves, training times, and inference times for various approaches in the experiment on interpolatory surface reconstruction with exact normal constraints. CNF outperforms prior work in terms of convergence, while maintaining similar training and inference efficiency.
Figure 16: Learning curves of various approaches for surface reconstruction with exact normal interpolation.
Table 7: Average training and inference time (in seconds) for surface reconstruction with exact normal interpolation.
Method	Training	Inference
Hypernet kernel	242	0.0663
SIREN [34]	148.4	0.1437
D Self-tuning PDE Solver
We selected 1D advection (described in Eq. 15 in Sec. 4.4) as a demonstration of the effectiveness of CNF in solving partial differential equations (PDEs). This choice was made due to its simple, linear form. Additionally, one of the advantages is that the ground truth function has a convenient analytical expression, eliminating the need for iterative solvers for validation. Specifically, our objective is to recover the following function:
$$
u (x, t) = \sin 2 \pi (x - \beta t), \quad x \in (0, 1), t \in (0, 1). \tag {22}
$$
D.2 Learning Curves
In Fig. 17 and Tab. 8, we report the learning curves, training times, and inference times for various approaches in the PDE experiment. We include SIREN [34] as an additional baseline for evaluation. CNF outperforms SIREN in terms of convergence, while maintaining similar training and inference efficiency.
Figure 17: Learning curves of various approaches for self-tuning PDE solver.
Table 8: Training and inference time (in seconds) of various approaches for self-tuning PDE solver.
Method	Training	Inference
SIREN [34]	36.7	0.0087
RBF	160.7	0.0056
Skewed RBF	265.6	0.0061
D.3 Transfer Learning
In Sec. 4.4, we detail how CNF can automatically fine-tune the hyperparameters of the skewed RBF on an irregular grid. Here, we illustrate that, given optimized hyperparameters of the skewed RBF for a particular grid, we can employ the resulting skewed RBF to directly solve a different advection equation (such as a different initial condition) at inference time, without additional training, if the grid remains the same.
For demonstration, we incrementally add a shift,  $\mu$ , to the initial condition of the advection equation, starting from small to large shifts:
$$
u _ {0} (x) = \sin (2 \pi x) + \mu , \quad x \in (0, 1). \tag {23}
$$
The hyperparameters of the skewed RBFs were trained solely when  $\mu = 0$ . When the initial condition,  $u_0(x)$ , is shifted, the new advection equation can be solved with the same kernel at inference time. No further training was conducted; see Tab. 9 for the results.
The efficacy of this approach can be explained by our perspective of formulating the constrained optimization problem as a collocation problem with learnable inductive bias. Once the preferred inductive bias away from the collocation points is obtained, solving a PDE at the same collocation points is reduced to a pure collocation problem, which requires minimal to no further training.
Table 9: Demonstration of transfer learning with the skewed RBF kernels. The kernels were initially trained exclusively for a 1D advection function with no shift (first row). Subsequently, their transfer learning capabilities were tested at varying shifts. No training was conducted other than for the first row.
Shift μ	Random init Skewed RBF	Pre-trained Skewed RBF
RMSE	nRMSE	RMSE	nRMSE
0	0.0127	0.0227	0.0070	0.0129
1	0.0460	0.0736	0.0049	0.0074
10	0.5788	0.0594	0.0177	0.0018
100	5.8613	0.0587	0.2221	0.0022
1000	60.143	0.0602	2.3113	0.0023
10000	635.78	0.0636	23.844	0.0024
Figure 9: Learned material appearance.
Figure 10: Learned material appearance.
Figure 11: Learned material appearance.
Figure 12: Learned material appearance.
Figure 13: Learned material appearance.
Figure 14: Learned material appearance.