rokuz7/nlg

Neural Light Grid generator

The generator network of a Neural Light Grid (Iwanicki et al., "Neural Light Grid", Advances in Real-Time Rendering in Games, SIGGRAPH 2024).

A Neural Light Grid is a lattice of irradiance probes. Every probe owns a weighting function over 8192 support points in the cube of one probe spacing around it: trilinear where the probe sees, diffused around corners through the visibility graph of the points, zero inside geometry. A small network per probe (3 → 8 → 8 → 1, LeakyReLU, 113 coefficients) stores that function. This model turns a function into the probe's network in one pass: the first 512 PCA coefficients of the function go through a 6-layer, 512-wide LeakyReLU MLP with skip connections that outputs the 113 coefficients.

Files

File	Size	Contents
nlg.vgf	6.1 MB	The generator, PCA included, as an int8 TOSA graph in a VGF container, for VK_ARM_data_graph
nlg.bin	15.1 MB	The support point pattern, the PCA, the float layers (for GPUs without data graphs) and the output scale of the graph

nlg.vgf

• Input: [1, 256, 1, 8192] int8, 256 probes per dispatch, every value round(255 * phi) - 128 of the probe's function at the 8192 support points (in the order of the pattern in nlg.bin).
• Output: [1, 256, 1, 113] int32 accumulators; coefficient i is output[i] * outputScale[i] (the last 113 floats of nlg.bin).
• Graph: the PCA projection as an int8 1x1 CONV2D with 8192 inputs (the mean folded into its bias), then every layer as an int8 CONV2D into int32 with a per-channel RESCALE, LeakyReLU (slope 0.01) as a TABLE, the skip connections summed in an int32 stream.

nlg.bin (little endian)

u32  header[8]       'NLGG' (0x47474C4E), version 2, points 8192, pca 512, width 512, layers 6, coefficients 113, 0
f32  points[8192][3] support points in [-1, 1]^3, in probe spacings
f32  mean[8192]      PCA mean
f16  basis[8192][512] PCA basis
     7 linear layers (first, 5 residual blocks, output), each:
f32  weightsT[inputs][outputs]   transpose of the PyTorch weight
f32  bias[outputs]
f32  outputScale[113]            scale of the int32 outputs of the graph

The generator computes h = leaky(first(pca(phi))), h += leaky(block(h)) five times, theta = last(h).

The 113 coefficients of a probe network

For a position x relative to the probe, in spacings: h1 = leaky(W1 x + b1), h2 = leaky(W2 h1 + b2), w = W3 h2 + b3, stored as W1[8][3] (row-major), b1[8], W2[8][8], b2[8], W3[8], b3.

Training

• Loss: the squared difference between the generated network and the function, at 2048 random valid support points per step, end to end through the 3-8-8-1 network. 40 epochs, Adam, cosine schedule.
• Quantization: per-channel post-training quantization, evaluated bit for bit against the integer arithmetic of the graph.

Limitations

• One lattice level and one probe spacing; no adaptive hierarchy as in the SIGGRAPH talk.
• The functions treat a point whose ray to a neighbour meets a back face as inside geometry, so scenes without closed or consistently oriented surfaces get fewer valid points.