RF: refactor check for nan_fill value

Make the check for nan_fill more direct; also, clip nan_fill with shared
range.