Repository: googleapis/google-cloud-python

Issue:
Blob.upload_from_file doesn't work for file-like objects without size parameter
When using `StringIO`, `io.BytesIO` or other file-like objects (such as Flask/werkzeug's file upload object), trying to upload via `Blob.upload_from_file` doesn't work because it attempts to find the size of the object using `os.stat` throwing a `AttributeError: StringIO instance has no attribute 'fileno'`. Passing in the size manually fixes this, but imo this inconsistency shouldn't burden the user as there are other ways than fileno to determine the size of a StringIO or io.BytesIO instance (`seek`, `tell`).

At any rate, the error message could be more helpful.



Discussion:
Can you give a stacktrace of the error message?

The offending line is

``` python
        # Get the basic stats about the file.
        total_bytes = size or os.fstat(file_obj.fileno()).st_size
```

which as @jonparrott points out, assumes the file-like object is a `file` (from the stdlib).

```
Traceback (most recent call last):
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__
    return self.wsgi_app(environ, start_response)
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/jonwayne/workspace/getting-started-python/env/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/jonwayne/workspace/getting-started-python/oauth2/__init__.py", line 33, in decr
    return f(*args, **kwargs)
  File "/Users/jonwayne/workspace/getting-started-python/bookshelf/crud.py", line 57, in add
    image_url = upload_image_file(request.files.get('image'))
  File "/Users/jonwayne/workspace/getting-started-python/bookshelf/crud.py", line 25, in upload_image_file
    current_app.config['ALLOWED_EXTENSIONS']
  File "/Users/jonwayne/workspace/getting-started-python/storage/__init__.py", line 52, in upload_file
    blob.upload_from_file(file)
  File "/Users/jonwayne/workspace/getting-started-python/env/src/gcloud/gcloud/storage/blob.py", line 395, in upload_from_file
    total_bytes = size or os.fstat(file_obj.fileno()).st_size
UnsupportedOperation: fileno
```

Ha, ok. Doesn't provide anything we didn't know.

Heh, let me know if there's anything else I can provide to help here. The file-like object is a flask file upload ([werkzeug.FileStorage](http://werkzeug.pocoo.org/docs/0.10/datastructures/#werkzeug.datastructures.FileStorage)). It fails likewise when using `blob.upload_from_file(file)` and `blob.upload_from_file(file.stream)`.


Resolving patch:
diff --git a/gcloud/storage/blob.py b/gcloud/storage/blob.py
--- a/gcloud/storage/blob.py
+++ b/gcloud/storage/blob.py
@@ -370,7 +370,8 @@ def upload_from_file(self, file_obj, rewind=False, size=None,
         :type size: int
         :param size: The number of bytes to read from the file handle.
                      If not provided, we'll try to guess the size using
-                     :func:`os.fstat`
+                     :func:`os.fstat`. (If the file handle is not from the
+                     filesystem this won't be possible.)
 
         :type content_type: string or ``NoneType``
         :param content_type: Optional type of content being uploaded.
@@ -382,6 +383,9 @@ def upload_from_file(self, file_obj, rewind=False, size=None,
                           ``NoneType``
         :param connection: Optional. The connection to use when sending
                            requests. If not provided, falls back to default.
+
+        :raises: :class:`ValueError` if size is not passed in and can not be
+                 determined
         """
         connection = _require_connection(connection)
         content_type = (content_type or self._properties.get('contentType') or
@@ -392,7 +396,13 @@ def upload_from_file(self, file_obj, rewind=False, size=None,
             file_obj.seek(0, os.SEEK_SET)
 
         # Get the basic stats about the file.
-        total_bytes = size or os.fstat(file_obj.fileno()).st_size
+        total_bytes = size
+        if total_bytes is None:
+            if hasattr(file_obj, 'fileno'):
+                total_bytes = os.fstat(file_obj.fileno()).st_size
+            else:
+                raise ValueError('total bytes could not be determined. Please '
+                                 'pass an explicit size.')
         headers = {
             'Accept': 'application/json',
             'Accept-Encoding': 'gzip, deflate',


Test patch:
diff --git a/gcloud/storage/test_blob.py b/gcloud/storage/test_blob.py
--- a/gcloud/storage/test_blob.py
+++ b/gcloud/storage/test_blob.py
@@ -410,6 +410,16 @@ def test_download_as_string(self):
         fetched = blob.download_as_string(connection=connection)
         self.assertEqual(fetched, b'abcdef')
 
+    def test_upload_from_file_size_failure(self):
+        BLOB_NAME = 'blob-name'
+        bucket = _Bucket()
+        blob = self._makeOne(BLOB_NAME, bucket=bucket)
+        file_obj = object()
+        connection = _Connection()
+        with self.assertRaises(ValueError):
+            blob.upload_from_file(file_obj, size=None,
+                                  connection=connection)
+
     def _upload_from_file_simple_test_helper(self, properties=None,
                                              content_type_arg=None,
                                              expected_content_type=None):