Add assert_not_NaN to find killer bugs where we lose numbers