ambient: Clamp sample values outside the landscape to the landscape border

GetPix() already has some handling for outside values, however that does not
always give the desired result.
Fix too large radius when converting radius from landscape to ambient coords

This is basically a leftover to do rounding when the radius was an integer
and not a floating point number.
ambient: fix some debug code
Use a named enum for the internalformat parameter of glTexImage2D