Cache some camera derived data for a frame.
Implement bilinear sampling of height values on the CPU side.