ENH: Add ability to create datasets using the gbq module
State: closed

Removed the `bq` command line module from `test_gbq.py`, as it doesn't support python 3. In order to do this, I had to create functions for `create_dataset()`, `delete_dataset()` and `dataset_exists()`.

This change is required for #11110

At the same time, I also implemented the following list functions: `list_dataset()` and `list_table()`.


Labels:

Comments:
The build failed on travis. I noticed an unusual error in the build log.
https://travis-ci.org/pydata/pandas/jobs/80567794

```
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "pandas/__init__.py", line 16, in <module>
    "extensions first.".format(module))
ImportError: C extension: hashtable not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.
nosetests --exe -A slow and not network and not disabled pandas --doctest-tests --with-xunit --xunit-file=/tmp/nosetests.xml
E
======================================================================
ERROR: Failure: ImportError (C extension: hashtable not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/miniconda/envs/pandas/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/travis/miniconda/envs/pandas/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/travis/miniconda/envs/pandas/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/travis/build/pydata/pandas/pandas/__init__.py", line 16, in <module>
    "extensions first.".format(module))
ImportError: C extension: hashtable not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace' to build the C extensions first.
```

@parthea I restarted the build. Maybe it was a one-off failure.

Ready for review. All tests passed in my local environment,

```
nosetests test_gbq.py -v
test_should_be_able_to_get_a_bigquery_service (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_results_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_schema_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_valid_credentials (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_make_a_connector (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_bad_project_id (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_bad_table_name (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order_plus_index (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_download_dataset_larger_than_200k_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_index_column (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_malformed_query (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_arbitrary_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_empty_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_false_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_timestamp_unix_epoch (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_true_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_unicode_string_conversion_and_normalization (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_zero_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_read_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_booleans_as_python_booleans (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_floats_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_integers_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_strings_as_python_strings (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_timestamps_as_numpy_datetime (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_that_parse_data_works_properly (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_should_fail_if_invalid_table_name_passed (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_create_bq_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_create_bq_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_exists (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_bq_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_bq_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_generate_bq_schema (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_google_upload_errors_should_raise_exception (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_bq_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_bq_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_bq_table_zero_results (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_table_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_append (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_fail (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_replace (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
pandas.io.tests.test_gbq.test_requirements ... ok

----------------------------------------------------------------------
Ran 53 tests in 368.008s
```

@parthea travis is fixed. go ahead and rebase

@jreback 
Could this make it into 0.17.0 ? It would helpful for my other PR #11110

what does create_dataset do? what is the difference between this and a table? 

list_table is ok.

why is this needed for #11110  ?

A table belongs to a dataset. In order to create a table, you must either have an existing dataset or create a new dataset.

Tables
https://cloud.google.com/bigquery/docs/reference/v2/tables

Datasets
https://cloud.google.com/bigquery/docs/reference/v2/datasets

From https://cloud.google.com/bigquery/what-is-bigquery#tables,
Datasets allow you to organize and control access to your tables. Because tables are contained in datasets, you'll need to create at least one dataset before loading data into BigQuery.

> why is this needed for #11110 ?

The `bq` command line module doesn't support python 3. The  `bq` command line module is currently used to create and delete datasets so unit tests will be broken in python 3 builds without the ability to create and delete datasets.

can you simply do this via a web interface? we are adding all of these functions which should be done externally to pandas.

separately I think we need to make the API's more explicit, e.g.

```
create_table
delete_table
table_exists
```

should these be

```
table_create
table_delete
table_exists
```

?

> can you simply do this via a web interface?

Yes, it can be done through web interface. From the integration testing point of view, it would be easier to be able to create datasets programatically. Each time the test starts, there is a newly created dataset used for testing. The dataset is deleted programatically after the test. 

One potential change could be to use the create and delete functions for unit testing purposes only without exposing the functionality globally.

```
table_create
table_delete
table_exists
```

Yes, I agree this is much better. I will commit a new version.

why don't we just advertise the `GbqConnector` as the main way to do things (and then you have methods you can call directly on it), e.g. `create()`, `delete()`, `exists()`. 

I would actually rename this to `Table`, then no problem with having an `Dataset` object as well.

e.g.

```
t = Table(project_id=....)
t.create()
t.delete()
```

sounds good! I'll commit a new version soon.

Ready for review. All tests passed.

```
nosetests test_gbq.py -v
test_should_be_able_to_get_a_bigquery_service (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_results_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_schema_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_valid_credentials (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_make_a_connector (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_bad_project_id (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_bad_table_name (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order_plus_index (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_download_dataset_larger_than_200k_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_index_column (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_malformed_query (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_arbitrary_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_empty_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_false_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_timestamp_unix_epoch (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_true_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_unicode_string_conversion_and_normalization (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_zero_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_read_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_booleans_as_python_booleans (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_floats_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_integers_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_strings_as_python_strings (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_timestamps_as_numpy_datetime (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_that_parse_data_works_properly (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_should_fail_if_invalid_table_name_passed (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_create_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_create_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_exists (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_generate_schema (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_google_upload_errors_should_raise_exception (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_table_zero_results (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_table_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_append (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_fail (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_replace (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
pandas.io.tests.test_gbq.test_requirements ... ok

----------------------------------------------------------------------
Ran 53 tests in 458.258s

OK
```

Travis is green

I am not familiar with gbq, but about the API: if you have a `gbq.Table` class, I would expect that this maps to a certain table in a gbq Dataset. But if that is the case, something like `table.exists('my_table')` is a bit strange. 
And given that a table gets created like `gbq.Table(projectid, 'my_dataset')`, this seems more like it maps to a Dataset, although you also have a `Dataset` class which does not map to a Dataset but only holds a project id.

@jorisvandenbossche 
I agree. I'm going to propose another naming convention. Assuming the following structure on Google BigQuery : 
1. A table belongs to a dataset. 
2. A dataset  belongs to a project id.

**Current implementation for Dataset class**
`dataset = gbq.Dataset(projectid)`
The `Dataset` class has the following instance methods:  `create`, `delete`, `list`, `exists`
- Where `create(datasetid)` creates the specified dataset under the project `projectid`.
- Where `delete(datasetid)` deletes the specified dataset under the project `projectid`.
- Where `list()` lists all datasets under the project `projectid`
- Where `exists(dataset_id)` checks to see if the dataset exists under project `projectid`

**Proposed implementation for Project, Dataset, Table class**

**Project class**

`project = gbq.Project(projectid)`
The `Project` class has the following instance methods: `dataset_list`
- Where `dataset_list()` lists all the datasets under project `projectid`

**Dataset class**

`dataset = gbq.Dataset(project, datasetid)`
The `Dataset` class has the following instance methods: `table_list`,`create`, `delete`, `exists`
- Where `table_list()` lists all tables under the dataset `datasetid`
- Where `create()` creates the specified dataset under the dataset `datasetid`.
- Where `delete()` deletes the specified dataset under the dataset `datasetid`.
- Where `exists()` checks to see if the dataset exists under dataset `datasetid`

**Table class**

`table = gbq.Table(dataset, tableid)`
The `Table` class has the following instance methods: `create`, `delete`, `exists`
- Where `create()` creates the specified table under the table `tableid`.
- Where `delete()` deletes the specified table under the table `tableid`.
- Where `exists()` checks to see if the table exists under table `tableid`

let's not add even more API here, keep it as minimal as possible.`Dataset/Table` are enough. and just have a `Dataset.tables()` method.

@parthea Is your assessment that this would be used a lot by users? (creating the DataSet 
programmatically) Or is it mainly because we need this ourselves for the tests? 
Because otherwise, we could maybe also start with just having these as private methods for internal use as a first step.

Further, creating the `Dataset` and `Table` objects to only be able to create or delete them, seems a bit overkill to me thinking a bit more about it.

```
dataset = gbq.Dataset(projectid, datasetid)
dataset.create()
```

vs

```
gbq.create_dataset(projectid, datasetid)
```

Object oriented APIs can be nice, but in some case just sticking to some functions can be simpler and more straightforward.

@jorisvandenbossche
@jreback

> we could maybe also start with just having these as private methods for internal use as a first step.

This need for this enhancement was driven around integration testing. While I was working on this enhancement, I thought it would be good to expose the create and delete functions for tables and datasets in case others require it. I believe the value of this enhancement is with integration testing so I am ok with using private functions to create and delete tables / datasets and see if there is demand to expose the functionality.

I think it would be more flexible for future development if I keep the `Table` and `Dataset` classes, but don't expose them. I could add the `dataset.tables()` instance method as requested. Listing datasets will also be useful for integration testing during the integration testing setup and teardown. (ie use `project.datasets()` to list datasets ). 

@parthea ok, let's keep them private for now (maybe make them `_Table`, `_Dataset` to recognize this, and take them out of the `API.rst`. All for integration testing :)

but since we are so close to actually releasing, just want to finish this up.

@jreback I am currently travelling. A colleague will commit the final changes for this PR (should be today)

@aaront 

@jreback let me know if anything else is needed

@parthea ok, small API change and a doc check. ping when green. (also confirm that this passes locally for you ).

@jreback made the requested changes except for the generator, I don't want to rock the boat too much at the moment :)

The indentation in api.rst looks OK on my end.

My only concern at this point is that some public API was removed: `create_table`, `delete_table`, and `table_exists` are no longer part of the module. If this is OK, then all is go.

I've just made a commit for some fixups for some broken tests, as well.

Running all the gbq tests locally yields:

```
> nosetests test_gbq.py -v
test_should_be_able_to_get_a_bigquery_service (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_results_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_schema_from_query (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_get_valid_credentials (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_should_be_able_to_make_a_connector (pandas.io.tests.test_gbq.TestGBQConnectorIntegration) ... ok
test_bad_project_id (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_bad_table_name (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_column_order_plus_index (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_download_dataset_larger_than_200k_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_index_column (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_malformed_query (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_arbitrary_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_empty_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_false_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_null_timestamp (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_timestamp_unix_epoch (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_true_boolean (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_floats (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_integers (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_should_properly_handle_valid_strings (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_unicode_string_conversion_and_normalization (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_zero_rows (pandas.io.tests.test_gbq.TestReadGBQIntegration) ... ok
test_read_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_booleans_as_python_booleans (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_floats_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_integers_as_python_floats (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_strings_as_python_strings (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_should_return_bigquery_timestamps_as_numpy_datetime (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_that_parse_data_works_properly (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_should_fail_if_invalid_table_name_passed (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_to_gbq_with_no_project_id_given_should_fail (pandas.io.tests.test_gbq.TestReadGBQUnitTests) ... ok
test_create_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_create_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_dataset_exists (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_delete_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_generate_schema (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_google_upload_errors_should_raise_exception (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_dataset (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_table (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_list_table_zero_results (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_table_does_not_exist (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_append (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_fail (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
test_upload_data_if_table_exists_replace (pandas.io.tests.test_gbq.TestToGBQIntegration) ... ok
pandas.io.tests.test_gbq.test_requirements ... ok

----------------------------------------------------------------------
Ran 53 tests in 429.681s

OK
```

Ping me if there's anything else needed.

cc: @parthea 

@aaront @parthea 

> My only concern at this point is that some public API was removed: create_table, delete_table, and table_exists are no longer part of the module. If this is OK, then all is go.

These only existed in master. They have not been released, so this is ok.

Just make the small change in the naming of `generate_schema` and I think good to go.

@jreback all 53 tests pass locally again.

Ping me if you'd like me to squash these commits. Will ping when travis is green.

Got your comment about deprecating & removing after I had made the rename of `generate_bq_schema`. Will remove this from the docs and properly deprecate.