@jtigani thanks for the comments.  We did consider those downsides, but in addition to the issues mentioned by @jacobschaer we wanted the implementation to be as simple and hopefully, therefore, as reliable as possible on our side.  

When we introduce temporary file objects it is adding complexity in terms of managing the object.  With HDFS dataframes, we thought we might still have to include chunking because we couldn't be sure the client would have memory for the entire object which then means multiple temporary file objects.  Additionally, we had to apply a workaround to the object to fake the mime types as bin like ```media_filename = media_file.name + '.bin'``` so that the upload would work given the API didn't work with the standard json file we created using the pandas methods. We'll look at the MediaInMemoryUpload as we hadn't looked at that, so that might help with some of this.

While the table is certainly required in this case, if we used the media upload, we would also have to handle table name collisions and the schema which adds more complexity. 

In addition to the downsides of our current approach you mentioned, we didn't like that you have two rate limiting classes of service- which is why we had to insert a 1 second sleep.

So, we were thinking we could keep to_gbq as experimental so we could hear more feedback on it before expanding the functionality.  Our design criteria was as to be as simple as possible while still delivering the functionality given we would get feedback.

We'll look at it again, so thanks for your feedback.  

@jreback In addition to the security of the credentials, there is also the issue of the BigQuery testing account we are using having costs associated with it.  We ideally wanted one that wasn't tied to my credit card.