Don't mess with NaN where possible, fixes #89