internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: googleapis/google-cloud-python

Issue:
Ability to cancel Cloud Bigtable reads
Cloud Bigtable scanning reads can be long.  In many cases, users want to cancel the request before the read stream is complete.  `read_rows` should return a cancelable generator.

This is an important feature for a Cloud Bigtable Apache Beam connector.


Discussion:

Resolving patch:
diff --git a/bigtable/google/cloud/bigtable/row_data.py b/bigtable/google/cloud/bigtable/row_data.py
--- a/bigtable/google/cloud/bigtable/row_data.py
+++ b/bigtable/google/cloud/bigtable/row_data.py
@@ -376,6 +376,9 @@ def __init__(self, read_method, request, retry=DEFAULT_RETRY_READ_ROWS):
         self.rows = {}
         self._state = self.STATE_NEW_ROW
 
+        # Flag to stop iteration, for any reason not related to self.retry()
+        self._cancelled = False
+
     @property
     def state(self):
         """State machine state.
@@ -388,6 +391,7 @@ def state(self):
 
     def cancel(self):
         """Cancels the iterator, closing the stream."""
+        self._cancelled = True
         self.response_iterator.cancel()
 
     def consume_all(self, max_loops=None):
@@ -436,7 +440,7 @@ def __iter__(self):
         Parse the response and its chunks into a new/existing row in
         :attr:`_rows`. Rows are returned in order by row key.
         """
-        while True:
+        while not self._cancelled:
             try:
                 response = self._read_next_response()
             except StopIteration:
@@ -445,6 +449,8 @@ def __iter__(self):
                 break
 
             for chunk in response.chunks:
+                if self._cancelled:
+                    break
                 self._process_chunk(chunk)
                 if chunk.commit_row:
                     self.last_scanned_row_key = self._previous_row.row_key


Test patch:
diff --git a/bigtable/tests/unit/test_row_data.py b/bigtable/tests/unit/test_row_data.py
--- a/bigtable/tests/unit/test_row_data.py
+++ b/bigtable/tests/unit/test_row_data.py
@@ -529,6 +529,7 @@ def test_cancel(self):
         self.assertEqual(response_iterator.cancel_calls, 0)
         yield_rows_data.cancel()
         self.assertEqual(response_iterator.cancel_calls, 1)
+        self.assertEqual(list(yield_rows_data), [])
 
     # 'consume_next' tested via 'TestPartialRowsData_JSON_acceptance_tests'
Repository: googleapis/google-cloud-python

Issue:
Storage: include generation in Blob repr.
I added a comment here based on the `generation` addition to `storage` https://github.com/googleapis/google-cloud-python/pull/7444#issuecomment-485874643

I think it's worth adding an issue too - specifically because now `generation` defines a blob but isn't included in the repr. Meaning you can get confusing results like this:

```python

blob
<Blob: sixty-capital-temp, tests/test_357256/blob_645776>

b2 = gcs.client(project).get_bucket(bucket).get_blob(blob.name)  # recreate 
b2
<Blob: sixty-capital-temp, tests/test_357256/blob_645776>  # same repr

blob.exists()
False

b2.exists()
True
```
... but b2 & blob have the same repr! Prior to the latest release, both results would have been `True`. 
If generation were in the repr, this would be less confusing - e.g. I wouldn't have had to read the release notes to work out what was going on



Discussion:

Resolving patch:
diff --git a/storage/google/cloud/storage/blob.py b/storage/google/cloud/storage/blob.py
--- a/storage/google/cloud/storage/blob.py
+++ b/storage/google/cloud/storage/blob.py
@@ -238,7 +238,7 @@ def __repr__(self):
         else:
             bucket_name = None
 
-        return "<Blob: %s, %s>" % (bucket_name, self.name)
+        return "<Blob: %s, %s, %s>" % (bucket_name, self.name, self.generation)
 
     @property
     def path(self):


Test patch:
Repository: googleapis/google-cloud-python

Issue:
Firestore: docs mentioning 'firestore.collection_group' are incorrect.
#### Environment details

# OS type and version
Mac OSX 10.14.5 

# Python version and virtual environment information:
python 3.7.3

#google-cloud-<service> version: 

google-api-core           1.12.0                   pypi_0    pypi
google-auth               1.6.3                    pypi_0    pypi
google-cloud-core         1.0.2                    pypi_0    pypi
google-cloud-firestore    1.2.0                    pypi_0    pypi
googleapis-common-protos  1.6.0

#### Steps to reproduce

1. Created a collection with documents and sub-collections
2. Created a composite index for collection group queries
3. Cannot figure out how to do a collection group query that returns any results

#### Code example

```python
from google.cloud import firestore
query = firestore.collection_group('mygroup') #directly from your docs
```

#### Stack trace
```
AttributeError: module 'google.cloud.firestore' has no attribute 'collection_group'
```

Thanks!



Discussion:
@szeitlin Thanks for the report!

Resolving patch:
diff --git a/firestore/google/cloud/firestore_v1/client.py b/firestore/google/cloud/firestore_v1/client.py
--- a/firestore/google/cloud/firestore_v1/client.py
+++ b/firestore/google/cloud/firestore_v1/client.py
@@ -203,7 +203,7 @@ def collection_group(self, collection_id):
 
         .. code-block:: python
 
-            >>> query = firestore.collection_group('mygroup')
+            >>> query = client.collection_group('mygroup')
 
         @param {string} collectionId Identifies the collections to query over.
         Every collection or subcollection with this ID as the last segment of its


Test patch:
Repository: googleapis/google-cloud-python

Issue:
Storage: Client.download_blob_to_file fails.
`Client.download_blob_to_file()` does not work properly when passing a URI because the parsed path which is passed as a blob name has the initial `/` .

https://github.com/googleapis/google-cloud-python/blob/master/storage/google/cloud/storage/client.py#L389-L395

It should be removed from the path before passing into a Blob constructor.
```
            scheme, netloc, path, query, frag = urlsplit(blob_or_uri)
            if scheme != "gs":
                raise ValueError("URI scheme must be gs")
            blob_name = path[1:]
            bucket = Bucket(self, name=netloc)
            blob_or_uri = Blob(blob_name, bucket)

            blob_or_uri.download_to_file(file_obj, client=self, start=start, end=end)
```


BTW: I tried to create a pull request to fix it but I was not allowed to push my commit into this repo. How can I contribute?


Discussion:
@shotarohorai Thank you for the report. Can you provide an example of a URI which does not work?

Our contributing guidelines are at https://github.com/googleapis/google-cloud-python/blob/master/CONTRIBUTING.rst. It's not expected that you would have write access to this repository, but you should be able to send a pull request from a fork.

Resolving patch:
diff --git a/storage/google/cloud/storage/client.py b/storage/google/cloud/storage/client.py
--- a/storage/google/cloud/storage/client.py
+++ b/storage/google/cloud/storage/client.py
@@ -390,7 +390,7 @@ def download_blob_to_file(self, blob_or_uri, file_obj, start=None, end=None):
             if scheme != "gs":
                 raise ValueError("URI scheme must be gs")
             bucket = Bucket(self, name=netloc)
-            blob_or_uri = Blob(path, bucket)
+            blob_or_uri = Blob(path[1:], bucket)
 
             blob_or_uri.download_to_file(file_obj, client=self, start=start, end=end)
 


Test patch:
diff --git a/storage/tests/system.py b/storage/tests/system.py
--- a/storage/tests/system.py
+++ b/storage/tests/system.py
@@ -552,6 +552,23 @@ def test_copy_existing_file(self):
         copied_contents = new_blob.download_as_string()
         self.assertEqual(base_contents, copied_contents)
 
+    def test_download_blob_w_uri(self):
+        blob = self.bucket.blob("MyBuffer")
+        file_contents = b"Hello World"
+        blob.upload_from_string(file_contents)
+        self.case_blobs_to_delete.append(blob)
+
+        temp_filename = tempfile.mktemp()
+        with open(temp_filename, "wb") as file_obj:
+            Config.CLIENT.download_blob_to_file(
+                "gs://" + self.bucket.name + "/MyBuffer", file_obj
+            )
+
+        with open(temp_filename, "rb") as file_obj:
+            stored_contents = file_obj.read()
+
+        self.assertEqual(file_contents, stored_contents)
+
 
 class TestUnicode(unittest.TestCase):
     @unittest.skipIf(RUNNING_IN_VPCSC, "Test is not VPCSC compatible.")
Repository: googleapis/google-cloud-python

Issue:
Firestore: sync shim module with 'firestore_v1' package.
See https://github.com/googleapis/google-cloud-python/issues/7533#issuecomment-495917295 &ff.


Discussion:

Resolving patch:
diff --git a/firestore/google/cloud/firestore.py b/firestore/google/cloud/firestore.py
--- a/firestore/google/cloud/firestore.py
+++ b/firestore/google/cloud/firestore.py
@@ -16,6 +16,8 @@
 
 
 from google.cloud.firestore_v1 import __version__
+from google.cloud.firestore_v1 import ArrayRemove
+from google.cloud.firestore_v1 import ArrayUnion
 from google.cloud.firestore_v1 import Client
 from google.cloud.firestore_v1 import CollectionReference
 from google.cloud.firestore_v1 import DELETE_FIELD
@@ -24,7 +26,10 @@
 from google.cloud.firestore_v1 import enums
 from google.cloud.firestore_v1 import ExistsOption
 from google.cloud.firestore_v1 import GeoPoint
+from google.cloud.firestore_v1 import Increment
 from google.cloud.firestore_v1 import LastUpdateOption
+from google.cloud.firestore_v1 import Maximum
+from google.cloud.firestore_v1 import Minimum
 from google.cloud.firestore_v1 import Query
 from google.cloud.firestore_v1 import ReadAfterWriteError
 from google.cloud.firestore_v1 import SERVER_TIMESTAMP
@@ -38,6 +43,8 @@
 
 __all__ = [
     "__version__",
+    "ArrayRemove",
+    "ArrayUnion",
     "Client",
     "CollectionReference",
     "DELETE_FIELD",
@@ -46,7 +53,10 @@
     "enums",
     "ExistsOption",
     "GeoPoint",
+    "Increment",
     "LastUpdateOption",
+    "Maximum",
+    "Minimum",
     "Query",
     "ReadAfterWriteError",
     "SERVER_TIMESTAMP",


Test patch:
diff --git a/firestore/tests/unit/test_firestore_shim.py b/firestore/tests/unit/test_firestore_shim.py
new file mode 100644
--- /dev/null
+++ b/firestore/tests/unit/test_firestore_shim.py
@@ -0,0 +1,29 @@
+# -*- coding: utf-8 -*-
+# Copyright 2017 Google LLC All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import unittest
+
+
+class TestFirestoreShim(unittest.TestCase):
+    def test_shim_matches_firestore_v1(self):
+        from google.cloud import firestore
+        from google.cloud import firestore_v1
+
+        self.assertEqual(firestore.__all__, firestore_v1.__all__)
+
+        for name in firestore.__all__:
+            found = getattr(firestore, name)
+            expected = getattr(firestore_v1, name)
+            self.assertIs(found, expected)
Repository: googleapis/google-cloud-python

Issue:
Core: PollingFuture.exception does not respect passed timeout
#### Environment details

2. OS type and version: Docker, ubuntu 18.04
3. Python version and virtual environment information: Python 3.7.1
4. google-cloud-bigquery version:  google-cloud-bigquery==1.9.0

#### Steps to reproduce

  1. When you e.g. run `QueryJob.exception(timeout=1)` it will run until job is done not respecting passed `timeout`, in the code https://github.com/googleapis/google-cloud-python/blob/951b9c68d2390eedd271b7eb1ca363cb48d82d8d/api_core/google/api_core/future/polling.py#L142 you can see that timeout argument is not passed to `self._blocking_poll()` as it is done in `self.result()`. Question is whether it is a bug or it is intentional?



Discussion:
@kamilglod Thanks for the report!

Resolving patch:
diff --git a/api_core/google/api_core/future/polling.py b/api_core/google/api_core/future/polling.py
--- a/api_core/google/api_core/future/polling.py
+++ b/api_core/google/api_core/future/polling.py
@@ -139,7 +139,7 @@ def exception(self, timeout=None):
             Optional[google.api_core.GoogleAPICallError]: The operation's
                 error.
         """
-        self._blocking_poll()
+        self._blocking_poll(timeout=timeout)
         return self._exception
 
     def add_done_callback(self, fn):


Test patch:
diff --git a/api_core/tests/unit/future/test_polling.py b/api_core/tests/unit/future/test_polling.py
--- a/api_core/tests/unit/future/test_polling.py
+++ b/api_core/tests/unit/future/test_polling.py
@@ -119,6 +119,12 @@ def test_result_timeout():
         future.result(timeout=1)
 
 
+def test_exception_timeout():
+    future = PollingFutureImplTimeout()
+    with pytest.raises(concurrent.futures.TimeoutError):
+        future.exception(timeout=1)
+
+
 class PollingFutureImplTransient(PollingFutureImplWithPoll):
     def __init__(self, errors):
         super(PollingFutureImplTransient, self).__init__()
Repository: googleapis/google-cloud-python

Issue:
Pub/Sub: logs littered with "The bidirectional RPC exited".
I am creating subscriptions to PubSub topics and reading messages from those subscriptions into queues. This is done interactively in a Jupyter notebook, so subscriptions can be created and destroyed repeatedly. In about ~20% of cases, I end up getting an error message logged to the console saying `ERROR:google.api_core.bidi:The bidirectional RPC exited.` However, I do not see that I am doing anything wrong?

Looking at the [code where the error messages are emitted](https://github.com/googleapis/google-cloud-python/blob/94043b1990433f0cfc78cd00fc6b135ef9291143/api_core/google/api_core/bidi.py#L673), it does not seem that anything pathological is going on. Maybe the severity of this error message should be reduced from `_LOGGER.error("The bidirectional RPC exited.")` to `_LOGGER.info(...)` or `_LOGGER.debug(...)`? 

#### Environment details

Debian GNU/Linux rodete
Python 2.7.15
google-cloud-core=1.0.2
google-cloud-pubsub=0.41.1

#### Steps to reproduce

Run the following code snippet multiple times. Some of the times we get the error message `ERROR:google.api_core.bidi:The bidirectional RPC exited`.

#### Code example

```python
import queue


def read_to_queue(subscription_path, topic_path):
    message_queue = queue.Queue()

    def callback(message):
        message.ack()
        message_queue.put(message.data)

    sub_client = pubsub_v1.SubscriberClient()
    _ = sub_client.create_subscription(subscription_path, topic_path)
    future = sub_client.subscribe(subscription_path, callback=callback)
    try:
        yield message_queue
    finally:
        future.cancel()
        sub_client.delete_subscription(subscription_path)
```


#### Stack trace

```
ERROR:google.api_core.bidi:The bidirectional RPC exited.
ERROR:google.api_core.bidi:The bidirectional RPC exited.
ERROR:google.api_core.bidi:The bidirectional RPC exited.
....
ERROR:google.api_core.bidi:The bidirectional RPC exited.
```



Discussion:
@ostrokach Thanks for the report.  It seems clear to me that the "error" here is *not* an error, but a result of a clean shutdown of the BiDi channel.

Resolving patch:
diff --git a/api_core/google/api_core/bidi.py b/api_core/google/api_core/bidi.py
--- a/api_core/google/api_core/bidi.py
+++ b/api_core/google/api_core/bidi.py
@@ -669,9 +669,6 @@ def _thread_main(self, ready):
                 exc,
             )
 
-        else:
-            _LOGGER.error("The bidirectional RPC exited.")
-
         _LOGGER.info("%s exiting", _BIDIRECTIONAL_CONSUMER_NAME)
 
     def start(self):


Test patch:
Repository: googleapis/google-cloud-python

Issue:
logging usage docs for list_entries method incorrect for both v1 and v2 library
#### Environment details

1. Logging API
2. OSX Mojave
3. Python version: 3.7.3
4. google-cloud-logging version:  
```
pip show google-cloud-logging
Name: google-cloud-logging
Version: 1.10.0
```

#### Steps to reproduce

  1. Navigate to the [usage start docs](https://googleapis.github.io/google-cloud-python/latest/logging/usage.html#retrieving-log-entries).
  2. Check out the example code `client.list_entries(project_ids=PROJECT_IDS)`
  3. Confirm that this is the [method name of the V1 library](https://github.com/googleapis/google-cloud-python/blob/master/logging/google/cloud/logging/client.py#L184-L186) mixed with the [renamed parameter of the v2 library](https://github.com/googleapis/google-cloud-python/blob/master/logging/google/cloud/logging_v2/gapic/logging_service_v2_client.py#L416-L419). 

I would have just cut a PR to fix it but it is unclear from that page whether the examples are for the v1 or v2 library.


Discussion:
@rsalmond For clarity, the method name you link to is not "the V1 library":  it is the high-level client object, which supports both GAPIC and HTTP/JSON requests to the V2 back-end API.

You are correct that the parameter name for the high-level client's `list_entries` method should be `projects`, rather than `project_ids`.
Here is the offending snippet:.
https://github.com/googleapis/google-cloud-python/blob/8581f6a118f13f1f4ce3131ea96fde7425c7bd4f/logging/docs/snippets.py#L104-L113

I need to figure out why those snippets aren't being tested under CI.

Resolving patch:
diff --git a/logging/docs/snippets.py b/logging/docs/snippets.py
--- a/logging/docs/snippets.py
+++ b/logging/docs/snippets.py
@@ -108,7 +108,7 @@ def client_list_entries_multi_project(
 
     # [START client_list_entries_multi_project]
     PROJECT_IDS = ["one-project", "another-project"]
-    for entry in client.list_entries(project_ids=PROJECT_IDS):  # API call(s)
+    for entry in client.list_entries(projects=PROJECT_IDS):  # API call(s)
         do_something_with(entry)
     # [END client_list_entries_multi_project]
 


Test patch:
Repository: googleapis/google-cloud-python

Issue:
BigQuery: Add enum with allowed data types for use in Schema
I received a feature request internally for an enum with allowed data types for use in SchemaField.

The string values should match the [names of the BigQuery legacy SQL data types](https://cloud.google.com/bigquery/data-types#legacy_sql_data_types ), but we could potentially add additional constants using the [standard SQL data type names](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types), but still the legacy SQL string value.


Discussion:
We might already have such a list of (legacy) data types somewhere? It'd be nice to have an enum for these, similar to how we have one for standard SQL now.
Standard SQL version: [google.cloud.bigquery.enums.StandardSqlDataTypes](https://googleapis.dev/python/bigquery/latest/generated/google.cloud.bigquery.enums.StandardSqlDataTypes.html)

Resolving patch:
diff --git a/bigquery/google/cloud/bigquery/enums.py b/bigquery/google/cloud/bigquery/enums.py
--- a/bigquery/google/cloud/bigquery/enums.py
+++ b/bigquery/google/cloud/bigquery/enums.py
@@ -67,3 +67,26 @@ def _make_sql_scalars_enum():
 
 
 StandardSqlDataTypes = _make_sql_scalars_enum()
+
+
+# See also: https://cloud.google.com/bigquery/data-types#legacy_sql_data_types
+# and https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types
+class SqlTypeNames(str, enum.Enum):
+    """Enum of allowed SQL type names in schema.SchemaField."""
+
+    STRING = "STRING"
+    BYTES = "BYTES"
+    INTEGER = "INTEGER"
+    INT64 = "INTEGER"
+    FLOAT = "FLOAT"
+    FLOAT64 = "FLOAT"
+    NUMERIC = "NUMERIC"
+    BOOLEAN = "BOOLEAN"
+    BOOL = "BOOLEAN"
+    GEOGRAPHY = "GEOGRAPHY"  # NOTE: not available in legacy types
+    RECORD = "RECORD"
+    STRUCT = "RECORD"
+    TIMESTAMP = "TIMESTAMP"
+    DATE = "DATE"
+    TIME = "TIME"
+    DATETIME = "DATETIME"


Test patch:
Repository: googleapis/google-cloud-python

Issue:
BigQuery: LoadJobConfig.schema setter should accept None
 **Is your feature request related to a problem? Please describe.**

There is no (public) way to unset a schema after you've set one. Even though the client doesn't distinguish between unset schema and an empty schema, the backend API does.

 **Describe the solution you'd like**

When the set schema is `None`, remove the whole schema property from the underlying `_properties`.

 **Describe alternatives you've considered**

We could always send `None` if a schema is empty, but this wouldn't be right, either.

 **Additional context**

Needed for https://github.com/googleapis/google-cloud-python/pull/9064



Discussion:
@plamut Should be a pretty small change to `LoadJobConfig`'s `@schema.setter` in `job.py`. Let me know if you can't get to this, and I'll try to tackle it tomorrow.
@tswast On it, prioritizing to unblock the PR #9064.

Resolving patch:
diff --git a/bigquery/google/cloud/bigquery/job.py b/bigquery/google/cloud/bigquery/job.py
--- a/bigquery/google/cloud/bigquery/job.py
+++ b/bigquery/google/cloud/bigquery/job.py
@@ -1160,6 +1160,10 @@ def schema(self):
 
     @schema.setter
     def schema(self, value):
+        if value is None:
+            self._del_sub_prop("schema")
+            return
+
         if not all(hasattr(field, "to_api_repr") for field in value):
             raise ValueError("Schema items must be fields")
         _helpers._set_sub_prop(


Test patch:
diff --git a/bigquery/tests/unit/test_job.py b/bigquery/tests/unit/test_job.py
--- a/bigquery/tests/unit/test_job.py
+++ b/bigquery/tests/unit/test_job.py
@@ -1504,6 +1504,19 @@ def test_schema_setter(self):
             config._properties["load"]["schema"], {"fields": [full_name_repr, age_repr]}
         )
 
+    def test_schema_setter_unsetting_schema(self):
+        from google.cloud.bigquery.schema import SchemaField
+
+        config = self._get_target_class()()
+        config._properties["load"]["schema"] = [
+            SchemaField("full_name", "STRING", mode="REQUIRED"),
+            SchemaField("age", "INTEGER", mode="REQUIRED"),
+        ]
+
+        config.schema = None
+        self.assertNotIn("schema", config._properties["load"])
+        config.schema = None  # no error, idempotent operation
+
     def test_schema_update_options_missing(self):
         config = self._get_target_class()()
         self.assertIsNone(config.schema_update_options)