Repository: googleapis/python-bigquery

Issue:
Client.insert_rows throws exception when coercing string to float from release 2.21.0
The behaviour of `Client.insert_rows` changed between release 2.20.0 and 2.21.0.

In 2.20.0, where the schema defines a field to be of type `FLOAT`, a string passed in would be coerced to a float in the resulting JSON.

In 2.21.0, where the schema defines a field to be of type `FLOAT`, a string passed in throws an exception.

I've tracked this down to #728 , where a check is added for `isnan` or `isinf`.  However, those functions require a float to be provided so throw an exception if a string is provided.  This check occurs before you get to the code that coerces the input to a float.

```python
def _float_to_json(value):
    """Coerce 'value' to an JSON-compatible representation."""
    if value is None:
        return None
    elif math.isnan(value) or math.isinf(value):
        return str(value)
    else:
        return float(value)
```

#### Environment details

  - OS type and version:
  - Python version: `Python 3.6.10`
  - pip version: `pip 21.2.1`
  - `google-cloud-bigquery` version: `2.22.1`

#### Steps to reproduce

  1. Setup table with a field defined to be `FLOAT`
  2. Pass a string to `Client.insert_rows()` for that field

#### Code example

```python
from google.cloud.bigquery import Client

client = Client(project='redacted')
client.insert_rows(table=client.get_table('python_test.test'), rows=[{'bar': "0.01"}])
```

#### Stack trace
```
  File "site-packages/google/cloud/bigquery/client.py", line 3329, in insert_rows
    json_rows = [_record_field_to_json(schema, row) for row in rows]
  File "site-packages/google/cloud/bigquery/client.py", line 3329, in <listcomp>
    json_rows = [_record_field_to_json(schema, row) for row in rows]
  File "site-packages/google/cloud/bigquery/_helpers.py", line 522, in _record_field_to_json
    record[subname] = _field_to_json(subfield, subvalue)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 589, in _field_to_json
    return _single_field_to_json(field, row_value)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 565, in _single_field_to_json
    return _scalar_field_to_json(field, row_value)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 464, in _scalar_field_to_json
    return converter(row_value)
  File "site-packages/google/cloud/bigquery/_helpers.py", line 345, in _float_to_json
    elif math.isnan(value) or math.isinf(value):
TypeError: must be real number, not str
```




Discussion:
Does `insert_rows_json` work for you? It doesn't do any type conversion.
I was able to reproduce this and it's indeed the change in in #728 that caused this.

Previously, the string values were simply converted to floats, but then extra checks were added to handle special float values (Nan, inf...). Unfortunately, those checks do not like strings.

I'll submit a fix soon.

Resolving patch:
diff --git a/google/cloud/bigquery/_helpers.py b/google/cloud/bigquery/_helpers.py
--- a/google/cloud/bigquery/_helpers.py
+++ b/google/cloud/bigquery/_helpers.py
@@ -19,6 +19,7 @@
 import decimal
 import math
 import re
+from typing import Union
 
 from google.cloud._helpers import UTC
 from google.cloud._helpers import _date_from_iso8601_date
@@ -338,14 +339,15 @@ def _int_to_json(value):
     return value
 
 
-def _float_to_json(value):
+def _float_to_json(value) -> Union[None, str, float]:
     """Coerce 'value' to an JSON-compatible representation."""
     if value is None:
         return None
-    elif math.isnan(value) or math.isinf(value):
-        return str(value)
-    else:
-        return float(value)
+
+    if isinstance(value, str):
+        value = float(value)
+
+    return str(value) if (math.isnan(value) or math.isinf(value)) else float(value)
 
 
 def _decimal_to_json(value):


Test patch:
diff --git a/tests/unit/test__helpers.py b/tests/unit/test__helpers.py
--- a/tests/unit/test__helpers.py
+++ b/tests/unit/test__helpers.py
@@ -690,21 +690,45 @@ def _call_fut(self, value):
     def test_w_none(self):
         self.assertEqual(self._call_fut(None), None)
 
+    def test_w_non_numeric(self):
+        with self.assertRaises(TypeError):
+            self._call_fut(object())
+
+    def test_w_integer(self):
+        result = self._call_fut(123)
+        self.assertIsInstance(result, float)
+        self.assertEqual(result, 123.0)
+
     def test_w_float(self):
         self.assertEqual(self._call_fut(1.23), 1.23)
 
+    def test_w_float_as_string(self):
+        self.assertEqual(self._call_fut("1.23"), 1.23)
+
     def test_w_nan(self):
         result = self._call_fut(float("nan"))
         self.assertEqual(result.lower(), "nan")
 
+    def test_w_nan_as_string(self):
+        result = self._call_fut("NaN")
+        self.assertEqual(result.lower(), "nan")
+
     def test_w_infinity(self):
         result = self._call_fut(float("inf"))
         self.assertEqual(result.lower(), "inf")
 
+    def test_w_infinity_as_string(self):
+        result = self._call_fut("inf")
+        self.assertEqual(result.lower(), "inf")
+
     def test_w_negative_infinity(self):
         result = self._call_fut(float("-inf"))
         self.assertEqual(result.lower(), "-inf")
 
+    def test_w_negative_infinity_as_string(self):
+        result = self._call_fut("-inf")
+        self.assertEqual(result.lower(), "-inf")
+
 
 class Test_decimal_to_json(unittest.TestCase):
     def _call_fut(self, value):