internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: googleapis/google-cloud-python

Issue:
storage's delete_bucket method should take a "force" parameter deleting even a full bucket
This would just be a shortcut for:

```
if force:
    for key in bucket:
        key.delete()
```

If people just want to kill a bucket, we shouldn't make it more complicated to do this.



Discussion:

Resolving patch:
diff --git a/gcloud/storage/bucket.py b/gcloud/storage/bucket.py
--- a/gcloud/storage/bucket.py
+++ b/gcloud/storage/bucket.py
@@ -130,7 +130,7 @@ def new_key(self, key):
 
     raise TypeError('Invalid key: %s' % key)
 
-  def delete(self):
+  def delete(self, force=False):
     """Delete this bucket.
 
     The bucket **must** be empty in order to delete it.
@@ -139,12 +139,20 @@ def delete(self):
     If the bucket is not empty,
     this will raise an Exception.
 
+    If you want to delete a non-empty bucket you can pass
+    in a force parameter set to true.
+    This will iterate through the bucket's keys and delete the related objects,
+    before deleting the bucket.
+
+    :type force: bool
+    :param full: If True, empties the bucket's objects then deletes it.
+
     :raises: :class:`gcloud.storage.exceptions.NotFoundError`
     """
 
     # TODO: Make sure the proper exceptions are raised.
 
-    return self.connection.delete_bucket(self.name)
+    return self.connection.delete_bucket(self.name, force=force)
 
   def delete_key(self, key):
     # TODO: Should we accept a 'silent' param here to not raise an exception?
diff --git a/gcloud/storage/connection.py b/gcloud/storage/connection.py
--- a/gcloud/storage/connection.py
+++ b/gcloud/storage/connection.py
@@ -340,7 +340,7 @@ def create_bucket(self, bucket, *args, **kwargs):
                                 data={'name': bucket.name})
     return Bucket.from_dict(response, connection=self)
 
-  def delete_bucket(self, bucket, *args, **kwargs):
+  def delete_bucket(self, bucket, force=False, *args, **kwargs):
     """Delete a bucket.
 
     You can use this method to delete a bucket by name,
@@ -369,12 +369,21 @@ def delete_bucket(self, bucket, *args, **kwargs):
     :type bucket: string or :class:`gcloud.storage.bucket.Bucket`
     :param bucket: The bucket name (or bucket object) to create.
 
+    :type force: bool
+    :param full: If True, empties the bucket's objects then deletes it.
+
     :rtype: bool
     :returns: True if the bucket was deleted.
     :raises: :class:`gcloud.storage.exceptions.NotFoundError`
     """
 
     bucket = self.new_bucket(bucket)
+
+    # This force delete operation is slow.
+    if force:
+      for key in bucket:
+        key.delete()
+
     response = self.api_request(method='DELETE', path=bucket.path)
     return True
 


Test patch:
Repository: googleapis/google-cloud-python

Issue:
Entities loaded in gcloud.datastore don't have a dataset
``` python
>>> dataset = demo.get_dataset()
>>> query = dataset.query()
>>> entity = query.fetch()[0]
>>> entity.delete()
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "gcloud/datastore/entity.py", line 206, in delete
    self.dataset().connection().delete_entity(
AttributeError: 'NoneType' object has no attribute 'delete_entity'
```

This is because we're creating entities from the protobufs, with the proper `dataset_id` but not a true reference to the Dataset object (which has a pointer to the connection).



Discussion:

Resolving patch:
diff --git a/gcloud/datastore/query.py b/gcloud/datastore/query.py
--- a/gcloud/datastore/query.py
+++ b/gcloud/datastore/query.py
@@ -244,4 +244,5 @@ def fetch(self, limit=None):
     entity_pbs = self.dataset().connection().run_query(
         query_pb=clone.to_protobuf(), dataset_id=self.dataset().id())
 
-    return [Entity.from_protobuf(entity) for entity in entity_pbs]
+    return [Entity.from_protobuf(entity, dataset=self.dataset())
+            for entity in entity_pbs]


Test patch:
Repository: googleapis/google-cloud-python

Issue:
AttributeError: url on Storage Exception when key not found
When attempting to get a key that does not exist the exception for the `NotFoundError` is trying to reference `request.url` which does not exist.

``` py
Traceback (most recent call last):
  [...]
    file_key = self.bucket.get_key(path)
  File "gcloud/storage/bucket.py", line 83, in get_key
    response = self.connection.api_request(method='GET', path=key.path)
  File "gcloud/storage/connection.py", line 212, in api_request
    raise exceptions.NotFoundError(response, content)
  File "gcloud/storage/exceptions.py", line 17, in __init__
    self.message = 'GET %s returned a 404.' % (response.url)
  File "httplib2/__init__.py", line 1680, in __getattr__
    raise AttributeError, name
AttributeError: url
```



Discussion:
Thanks for this report. I'm having a look.


Resolving patch:
diff --git a/gcloud/storage/exceptions.py b/gcloud/storage/exceptions.py
--- a/gcloud/storage/exceptions.py
+++ b/gcloud/storage/exceptions.py
@@ -14,7 +14,7 @@ def __init__(self, response, content):
 class NotFoundError(ConnectionError):
 
   def __init__(self, response, content):
-    self.message = 'GET %s returned a 404.' % (response.url)
+    self.message = 'Request returned a 404. Headers: %s' % (response)
 
 
 class StorageDataError(StorageError):


Test patch:
diff --git a/gcloud/storage/test_connection.py b/gcloud/storage/test_connection.py
--- a/gcloud/storage/test_connection.py
+++ b/gcloud/storage/test_connection.py
@@ -1,10 +1,19 @@
 import unittest2
 
 from gcloud.storage.connection import Connection
-
+from gcloud.storage.exceptions import NotFoundError
 
 class TestConnection(unittest2.TestCase):
 
   def test_init(self):
     connection = Connection('project-name')
     self.assertEqual('project-name', connection.project)
+
+
+class TestExceptions(unittest2.TestCase):
+
+  def test_not_found_always_prints(self):
+    e = NotFoundError({}, None)
+    self.assertEqual('', str(e))
+
+
Repository: googleapis/google-cloud-python

Issue:
gcloud.datastore.helpers.get_value_from_protobuf() returns naive datetimes
Naive datetimes (without timezones) are explicitly coerced to UTC by
'get_protobuf_attribute_and_value':

```
>>> import calendar
>>> import datetime
>>> import pytz
>>> from gcloud.datastore.helpers import get_protobuf_attribute_and_value
>>> naive = datetime.datetime(2014, 9, 16, 10, 19, 32, 4375) # no zone
>>> utc = datetime.datetime(2014, 9, 16, 10, 19, 32, 4375, pytz.utc)
>>> _, naive_micros = get_protobuf_attribute_and_value(naive)
>>> _, utc_micros = get_protobuf_attribute_and_value(utc)
>>> naive_micros == utc_micros
True
```

But the datetimes returned by 'get_value_from_protobuf' do not set the UTC
timezone:

```
>>> from gcloud.datastore.helpers import get_value_from_protobuf
>>> from gcloud.datastore.datastore_v1_pb2 import Property
>>> pb = Property()
>>> pb.value.timestamp_microseconds_value = utc_micros
>>> get_value_from_protobuf(pb) == utc
TypeError: can't compare offset-naive and offset-aware datetimes
>>> get_value_from_protobuf(pb) == naive
True
```

Leaving them naive when we know they are UTC seens wrong:  is there a policy in place which says we will keep in-memory timestamps around as naive-but-UTC-by-convention?



Discussion:

Resolving patch:
diff --git a/gcloud/datastore/helpers.py b/gcloud/datastore/helpers.py
--- a/gcloud/datastore/helpers.py
+++ b/gcloud/datastore/helpers.py
@@ -78,8 +78,9 @@ def get_value_from_protobuf(pb):
 
   if pb.value.HasField('timestamp_microseconds_value'):
     microseconds = pb.value.timestamp_microseconds_value
-    return (datetime.utcfromtimestamp(0) +
-            timedelta(microseconds=microseconds))
+    naive = (datetime.utcfromtimestamp(0) +
+             timedelta(microseconds=microseconds))
+    return naive.replace(tzinfo=pytz.utc)
 
   elif pb.value.HasField('key_value'):
     return Key.from_protobuf(pb.value.key_value)


Test patch:
diff --git a/gcloud/datastore/test_helpers.py b/gcloud/datastore/test_helpers.py
--- a/gcloud/datastore/test_helpers.py
+++ b/gcloud/datastore/test_helpers.py
@@ -92,9 +92,7 @@ def test_datetime(self):
         utc = datetime.datetime(2014, 9, 16, 10, 19, 32, 4375, pytz.utc)
         micros = (calendar.timegm(utc.timetuple()) * 1000000) + 4375
         pb = self._makePB('timestamp_microseconds_value', micros)
-        # self.assertEqual(self._callFUT(pb), utc) XXX
-        # see https://github.com/GoogleCloudPlatform/gcloud-python/issues/131
-        self.assertEqual(self._callFUT(pb), naive)
+        self.assertEqual(self._callFUT(pb), utc)
 
     def test_key(self):
         from gcloud.datastore.datastore_v1_pb2 import Property
Repository: googleapis/google-cloud-python

Issue:
gcloud.datastore.helpers.get_protobuf_attribute_and_value() allows out-of-range long integers
If I construct an integer too long to encode as a 64-bit integer, get_protobuf_attribute_and_value() does not raise a `ValueError`:

```
>>> gcloud.datastore.helpers.get_protobuf_attribute_and_value
>>> too_long = 1 << 63
>>> get_protobuf_attribute_and_value(too_long)
('integer_value', 9223372036854775808L)
```

even though that value is not usable for an 'integer_value' field:

```
>>> from gcloud.datastore.datastore_v1_pb2 import Property
>>> pb = Property()
>>> pb.value.integer_value = too_long
...
File ".../lib/python2.7/site-packages/google/protobuf/internal/type_checkers.py", line 119, in CheckValue
    raise ValueError('Value out of range: %d' % proposed_value)
ValueError: Value out of range: 9223372036854775808
```



Discussion:

Resolving patch:
diff --git a/gcloud/datastore/helpers.py b/gcloud/datastore/helpers.py
--- a/gcloud/datastore/helpers.py
+++ b/gcloud/datastore/helpers.py
@@ -2,11 +2,15 @@
 import calendar
 from datetime import datetime, timedelta
 
+from google.protobuf.internal.type_checkers import Int64ValueChecker
 import pytz
 
 from gcloud.datastore.key import Key
 
 
+INT64 = Int64ValueChecker().CheckValue
+
+
 def get_protobuf_attribute_and_value(val):
   """Given a value, return the protobuf attribute name and proper value.
 
@@ -53,7 +57,7 @@ def get_protobuf_attribute_and_value(val):
   elif isinstance(val, float):
     name, value = 'double', val
   elif isinstance(val, (int, long)):
-    name, value = 'integer', val
+    name, value = 'integer', INT64(val)
   elif isinstance(val, basestring):
     name, value = 'string', val
 


Test patch:
diff --git a/gcloud/datastore/test_helpers.py b/gcloud/datastore/test_helpers.py
--- a/gcloud/datastore/test_helpers.py
+++ b/gcloud/datastore/test_helpers.py
@@ -56,11 +56,19 @@ def test_int(self):
         self.assertEqual(value, 42)
 
     def test_long(self):
-        must_be_long = 1 << 63
+        must_be_long = (1 << 63) - 1
         name, value = self._callFUT(must_be_long)
         self.assertEqual(name, 'integer_value')
         self.assertEqual(value, must_be_long)
 
+    def test_long_too_small(self):
+        too_small = -(1 << 63) - 1
+        self.assertRaises(ValueError, self._callFUT, too_small)
+
+    def test_long_too_large(self):
+        too_large = 1 << 63
+        self.assertRaises(ValueError, self._callFUT, too_large)
+
     def test_native_str(self):
         name, value = self._callFUT('str')
         self.assertEqual(name, 'string_value')
Repository: googleapis/google-cloud-python

Issue:
gcloud.datastore.key.Key.from_protobuf drops namespace



Discussion:

Resolving patch:
diff --git a/gcloud/datastore/key.py b/gcloud/datastore/key.py
--- a/gcloud/datastore/key.py
+++ b/gcloud/datastore/key.py
@@ -68,8 +68,11 @@ def from_protobuf(cls, pb, dataset=None):
 
     if not dataset:
       dataset = Dataset(id=pb.partition_id.dataset_id)
+      namespace = pb.partition_id.namespace
+    else:
+      namespace = None
 
-    return cls(path=path, dataset=dataset)
+    return cls(dataset, namespace, path)
 
   def to_protobuf(self):
     """Return a protobuf corresponding to the key.


Test patch:
diff --git a/gcloud/datastore/test_key.py b/gcloud/datastore/test_key.py
--- a/gcloud/datastore/test_key.py
+++ b/gcloud/datastore/test_key.py
@@ -80,12 +80,20 @@ def test_from_protobuf_w_dataset_in_pb(self):
         key = self._getTargetClass().from_protobuf(pb)
         self.assertEqual(key.dataset().id(), _DATASET)
 
-    def test_from_protobuf_w_namespace_in_pb(self):
+    def test_from_protobuf_w_namespace_in_pb_wo_dataset_passed(self):
         _NAMESPACE = 'NAMESPACE'
         pb = self._makePB(namespace=_NAMESPACE)
         key = self._getTargetClass().from_protobuf(pb)
-        # See: https://github.com/GoogleCloudPlatform/gcloud-python/issues/133
-        # self.assertEqual(key.namespace(), _NAMESPACE)  XXX
+        self.assertEqual(key.namespace(), _NAMESPACE)
+
+    def test_from_protobuf_w_namespace_in_pb_w_dataset_passed(self):
+        from gcloud.datastore.dataset import Dataset
+        _DATASET = 'DATASET'
+        _NAMESPACE = 'NAMESPACE'
+        dataset = Dataset(_DATASET)
+        pb = self._makePB(namespace=_NAMESPACE)
+        key = self._getTargetClass().from_protobuf(pb, dataset)
+        self.assertEqual(key.namespace(), None)
 
     def test_from_protobuf_w_path_in_pb(self):
         _DATASET = 'DATASET'
Repository: googleapis/google-cloud-python

Issue:
gcloud.datastore.key.Key.from_path is undocumented, untested
I'm not sure what its contract is, either:

```
>>> from gcloud.datastore.key import Key
>>> Key.from_path().path()
[{'kind': ''}]
>>> Key.from_path('abc').path()
[{'kind': ''}]
>>> Key.from_path('abc', 'def').path()
[{'kind': 'abc', 'name': 'def'}]
>>> Key.from_path('abc', 'def', 'ghi').path()
[{'kind': 'abc', 'name': 'def'}]
>>> Key.from_path('abc', 'def', 'ghi', 123).path()
[{'kind': 'abc', 'name': 'def'}, {'kind': 'ghi', 'id': 123}]
```

Dropping the odd last element doesn't seem correct.



Discussion:

Resolving patch:
diff --git a/gcloud/datastore/key.py b/gcloud/datastore/key.py
--- a/gcloud/datastore/key.py
+++ b/gcloud/datastore/key.py
@@ -122,6 +122,9 @@ def from_path(cls, *args, **kwargs):
     :rtype: :class:`gcloud.datastore.key.Key`
     :returns: a new `Key` instance
     """
+    if len(args) % 2:
+        raise ValueError('Must pass an even number of args.')
+
     path = []
     items = iter(args)
 


Test patch:
diff --git a/gcloud/datastore/test_key.py b/gcloud/datastore/test_key.py
--- a/gcloud/datastore/test_key.py
+++ b/gcloud/datastore/test_key.py
@@ -164,12 +164,11 @@ def test_from_path_empty(self):
         self.assertEqual(key.path(), [{'kind': ''}])
 
     def test_from_path_single_element(self):
-        # See https://github.com/GoogleCloudPlatform/gcloud-python/issues/134
-        key = self._getTargetClass().from_path('abc')
-        self.assertEqual(key.dataset(), None)
-        self.assertEqual(key.namespace(), None)
-        self.assertEqual(key.kind(), '') # XXX s.b. 'abc'?
-        self.assertEqual(key.path(), [{'kind': ''}]) # XXX s.b. 'abc'?
+        self.assertRaises(ValueError, self._getTargetClass().from_path, 'abc')
+
+    def test_from_path_three_elements(self):
+        self.assertRaises(ValueError, self._getTargetClass().from_path,
+                                                    'abc', 'def', 'ghi')
 
     def test_from_path_two_elements_second_string(self):
         key = self._getTargetClass().from_path('abc', 'def')
Repository: googleapis/google-cloud-python

Issue:
gcloud.datastore.key.Key.parent untested, undocumented, useless
At the moment, it just raises `NotImplementedError`.  I imagine that it is
intended to return a new key whose path is the "parent path" of the
current key.  E.g.:

```
>>> from gcloud.datastore.key import Key
>>> PATH = [{'kind': 'parent', 'id': 123}, {'kind': 'child', 'id': 456}]
>>> child = Key(path=PATH)
>>> parent = key.parent()
>>> parent.path() == child.path[:-1]
True
```



Discussion:

Resolving patch:
diff --git a/gcloud/datastore/key.py b/gcloud/datastore/key.py
--- a/gcloud/datastore/key.py
+++ b/gcloud/datastore/key.py
@@ -33,6 +33,9 @@ def _clone(self):
     We make a shallow copy of the :class:`gcloud.datastore.dataset.Dataset`
     because it holds a reference an authenticated connection,
     which we don't want to lose.
+
+    :rtype: :class:`gcloud.datastore.key.Key`
+    :returns: a new `Key` instance
     """
     clone = copy.deepcopy(self)
     clone._dataset = self._dataset  # Make a shallow copy of the Dataset.
@@ -259,8 +262,15 @@ def id_or_name(self):
     return self.id() or self.name()
 
   def parent(self):#pragma NO COVER
-    # See https://github.com/GoogleCloudPlatform/gcloud-python/issues/135
-    raise NotImplementedError
+    """Getter:  return a new key for the next highest element in path.
+
+    :rtype: :class:`gcloud.datastore.key.Key`
+    :returns: a new `Key` instance, whose path consists of all but the last
+    element of self's path.  If self has only one path element, return None.
+    """
+    if len(self._path) <= 1:
+        return None
+    return self.path(self.path()[:-1])
 
   def __repr__(self): #pragma NO COVER
     return '<Key%s>' % self.path()


Test patch:
diff --git a/gcloud/datastore/test_key.py b/gcloud/datastore/test_key.py
--- a/gcloud/datastore/test_key.py
+++ b/gcloud/datastore/test_key.py
@@ -339,6 +339,14 @@ def test_id_or_name_w_name_only(self):
         key = self._makeOne(path=_PATH)
         self.assertEqual(key.id_or_name(), _NAME)
 
-    def _ugh(self):
-        protokey = key.to_protobuf()
-        self.assertEqual(protokey.partition_id.dataset_id, _DATASET)
+    def test_parent_default(self):
+        key = self._makeOne()
+        self.assertEqual(key.parent(), None)
+
+    def test_parent_explicit_top_level(self):
+        key = self._getTargetClass().from_path('abc', 'def')
+        self.assertEqual(key.parent(), None)
+
+    def test_parent_explicit_nested(self):
+        key = self._getTargetClass().from_path('abc', 'def', 'ghi', 123)
+        self.assertEqual(key.parent().path(), [{'kind': 'abc', 'name': 'def'}])
Repository: googleapis/google-cloud-python

Issue:
gcloud.storage.bucket.Bucket.upload_file defaults 'key' to None
The API cannot do anything useful with the default:  either it should be a required argument, or else the method should do something sensible if it is not passed.  E.g., one might crack the last element of the filename off and use that as a default key.



Discussion:
The docstring actually requires the latter choice:

https://github.com/GoogleCloudPlatform/gcloud-python/blob/1f2aaf3606802f4f02eeb30e22e1082ef7f4594e/gcloud/storage/bucket.py#L234


Resolving patch:
diff --git a/gcloud/storage/bucket.py b/gcloud/storage/bucket.py
--- a/gcloud/storage/bucket.py
+++ b/gcloud/storage/bucket.py
@@ -1,3 +1,5 @@
+import os
+
 from gcloud.storage import exceptions
 from gcloud.storage.acl import BucketACL
 from gcloud.storage.acl import DefaultObjectACL
@@ -230,6 +232,8 @@ def upload_file(self, filename, key=None):
                 to the root of the bucket
                 with the same name as on your local file system.
     """
+    if key is None:
+        key = os.path.basename(filename)
     key = self.new_key(key)
     return key.set_contents_from_filename(filename)
 


Test patch:
diff --git a/gcloud/storage/test_bucket.py b/gcloud/storage/test_bucket.py
--- a/gcloud/storage/test_bucket.py
+++ b/gcloud/storage/test_bucket.py
@@ -246,8 +246,22 @@ def test_delete_keys_miss(self):
         self.assertEqual(kw[1]['method'], 'DELETE')
         self.assertEqual(kw[1]['path'], '/b/%s/o/%s' % (NAME, NONESUCH))
 
-    # See: https://github.com/GoogleCloudPlatform/gcloud-python/issues/137
-    #def test_upload_file_default_key(self):
+    def test_upload_file_default_key(self):
+        from gcloud.test_credentials import _Monkey
+        from gcloud.storage import bucket as MUT
+        BASENAME = 'file.ext'
+        FILENAME = '/path/to/%s' % BASENAME
+        _uploaded = []
+        class _Key(object):
+            def __init__(self, bucket, name):
+                self._bucket = bucket
+                self._name = name
+            def set_contents_from_filename(self, filename):
+                _uploaded.append((self._bucket, self._name, filename))
+        bucket = self._makeOne()
+        with _Monkey(MUT, Key=_Key):
+            bucket.upload_file(FILENAME)
+        self.assertEqual(_uploaded, [(bucket, BASENAME, FILENAME)])
 
     def test_upload_file_explicit_key(self):
         from gcloud.test_credentials import _Monkey
Repository: googleapis/google-cloud-python

Issue:
gcloud.storage.connection.Connection.get_bucket takes spurious '*args', '**kw' arguments
... but ignores them:

https://github.com/GoogleCloudPlatform/gcloud-python/blob/1f2aaf3606802f4f02eeb30e22e1082ef7f4594e/gcloud/storage/connection.py#L258

Likewise the 'create_bucket':

https://github.com/GoogleCloudPlatform/gcloud-python/blob/1f2aaf3606802f4f02eeb30e22e1082ef7f4594e/gcloud/storage/connection.py#L319

and 'delete_bucket':

https://github.com/GoogleCloudPlatform/gcloud-python/blob/1f2aaf3606802f4f02eeb30e22e1082ef7f4594e/gcloud/storage/connection.py#L343



Discussion:

Resolving patch:
diff --git a/gcloud/storage/connection.py b/gcloud/storage/connection.py
--- a/gcloud/storage/connection.py
+++ b/gcloud/storage/connection.py
@@ -238,7 +238,7 @@ def api_request(self, method, path, query_params=None,
 
     return content
 
-  def get_all_buckets(self, *args, **kwargs):
+  def get_all_buckets(self):
     """Get all buckets in the project.
 
     This will not populate the list of keys available
@@ -261,7 +261,7 @@ def get_all_buckets(self, *args, **kwargs):
 
     return list(self)
 
-  def get_bucket(self, bucket_name, *args, **kwargs):
+  def get_bucket(self, bucket_name):
     """Get a bucket by name.
 
     If the bucket isn't found,
@@ -320,7 +320,7 @@ def lookup(self, bucket_name):
     except exceptions.NotFoundError:
       return None
 
-  def create_bucket(self, bucket, *args, **kwargs):
+  def create_bucket(self, bucket):
     """Create a new bucket.
 
     For example::
@@ -343,7 +343,7 @@ def create_bucket(self, bucket, *args, **kwargs):
                                 data={'name': bucket.name})
     return Bucket.from_dict(response, connection=self)
 
-  def delete_bucket(self, bucket, force=False, *args, **kwargs):
+  def delete_bucket(self, bucket, force=False):
     """Delete a bucket.
 
     You can use this method to delete a bucket by name,


Test patch: