Repository: googleapis/google-cloud-python

Issue:
'Table.insert_data()' fails when schema is not loaded
I tried to insert data using the google cloud python client but seem to get rows where every field has a null value. I used the [code in the latest documentation](https://googlecloudplatform.github.io/google-cloud-python/stable/bigquery-usage.html).

Using Debian 7.11 with Python 2.7.3 and google-cloud-python 0.20.



Discussion:
For clarity: [this stackoverflow question](http://stackoverflow.com/questions/39678654/uploading-empty-data-to-bigquery-using-gcloud-python) refers to the same problem.

If I insert rows using the following code:

``` python
client = bigquery.Client()
dataset = client.dataset('test')
table = dataset.table("test_table")
rows = [("foo", "bar"), ("foo2", "bar2")]
result = table.insert_data(rows)
```

and then query the latest upload, I get 

```
[(None, None), (None, None)]
```

@rishsriv Thanks for filing!

@rishsriv We have a working [system test which exercises `Table.insert_data`](https://github.com/GoogleCloudPlatform/google-cloud-python/blob/938ac216646c62acab8e5aef7dab33c911dbef1b/system_tests/bigquery.py#L263).  Does that example help?

Do the dataset and table already exist before your start?  If so, does the table's schema match the data you are inserting?

@tseaver Yes they do. And yes, the table's schema matches the data exactly.

``` python
>>> from google.cloud.bigquery import Client
>>> from google.cloud.bigquery.schema import SchemaField
>>> client = Client()
>>> dataset = client.dataset('gcp_2472')
>>> dataset.exists()
False
>>> dataset.create()
>>> left = SchemaField('left', 'STRING', 'REQUIRED')
>>> right = SchemaField('right', 'STRING', 'REQUIRED')
>>> table = dataset.table('test_table', schema=[left, right])
>>> table.create()
>>> rows = [("foo", "bar"), ("foo2", "bar2")]
>>> table.insert_data(rows); fetched, total, token = table.fetch_data()
[]
>>> fetched
[]
>>> fetched, total, token = table.fetch_data()
>>> fetched
[('foo', 'bar'), ('foo2', 'bar2')]
```

So, the rows weren't present at all for me when querying immediately after the call to `insert_data`, but they became available shortly thereafter.  The [BigQuery docs](https://cloud.google.com/bigquery/streaming-data-into-bigquery#dataavailability) say:

> Streamed data is available for real-time analysis within a few seconds of the first streaming insertion into a table.
> 
> Data can take up to 90 minutes to become available for copy and export operations

@tseaver Thank you for the super quick response. The problem for me was that the response (`fetched`) was not an empty list, but rather a list like this 

```
[(None, None), (None, None)]
```

Will try again to make sure I did not do something wrong.

@tseaver: Resolved! I had created the table using the webview, and simply used `table = dataset.table("test_table")` when declaring the table.

Explicitly declaring the schema (i.e., using the following lines)

```
left = SchemaField('left', 'STRING', 'REQUIRED')
right = SchemaField('right', 'STRING', 'REQUIRED')
table = dataset.table('test_table', schema=[left, right])
```

helped fix the problem.

Thank you very much for your time.

@rishsriv In your case, calling `table.reload()` (to fetch the schema) before `table.insert_data(rows)` would have worked as well. I have reopened the issue because `Table.insert_rows` should detect that `self._schema` is not populated, and raise an exception.


Resolving patch:
diff --git a/bigquery/google/cloud/bigquery/table.py b/bigquery/google/cloud/bigquery/table.py
--- a/bigquery/google/cloud/bigquery/table.py
+++ b/bigquery/google/cloud/bigquery/table.py
@@ -35,6 +35,7 @@
 from google.cloud.bigquery._helpers import _rows_from_json
 
 
+_TABLE_HAS_NO_SCHEMA = "Table has no schema:  call 'table.reload()'"
 _MARKER = object()
 
 
@@ -736,7 +737,11 @@ def insert_data(self,
                   identifies the row, and the "errors" key contains a list
                   of the mappings describing one or more problems with the
                   row.
+        :raises: ValueError if table's schema is not set
         """
+        if len(self._schema) == 0:
+            raise ValueError(_TABLE_HAS_NO_SCHEMA)
+
         client = self._require_client(client)
         rows_info = []
         data = {'rows': rows_info}


Test patch:
diff --git a/bigquery/unit_tests/test_table.py b/bigquery/unit_tests/test_table.py
--- a/bigquery/unit_tests/test_table.py
+++ b/bigquery/unit_tests/test_table.py
@@ -1248,6 +1248,23 @@ def test_fetch_data_w_record_schema(self):
         self.assertEqual(req['method'], 'GET')
         self.assertEqual(req['path'], '/%s' % PATH)
 
+    def test_insert_data_wo_schema(self):
+        from google.cloud.bigquery.table import _TABLE_HAS_NO_SCHEMA
+        client = _Client(project=self.PROJECT)
+        dataset = _Dataset(client)
+        table = self._makeOne(self.TABLE_NAME, dataset=dataset)
+        ROWS = [
+            ('Phred Phlyntstone', 32),
+            ('Bharney Rhubble', 33),
+            ('Wylma Phlyntstone', 29),
+            ('Bhettye Rhubble', 27),
+        ]
+
+        with self.assertRaises(ValueError) as exc:
+            table.insert_data(ROWS)
+
+        self.assertEqual(exc.exception.args, (_TABLE_HAS_NO_SCHEMA,))
+
     def test_insert_data_w_bound_client(self):
         import datetime
         from google.cloud._helpers import UTC