Repository: googleapis/google-cloud-python

Issue:
Datastore: broken paging next_page_token
I want to use `datastore` paging to fetch the query results. Specifically, I upload 11 entities to `datastore` and then want to fetch them (`limit=10`) using following code:
```python
query = <google.cloud.datastore.Client>.query(kind='test_kind')

query_iter = query.fetch(limit=limit, start_cursor=cursor)
page = next(query_iter.pages)
next_cursor = query_iter.next_page_token
entities = list(page)

logging.info('Cursor: %s, Next cursor: %s, Loaded entities: %d.',
              cursor, next_cursor, len(entities))
```
(where initially `cursor = None` and then is set to `next_cursor` value)

Unfortunately, I am getting not `None` value for the `next_cursor` even there are not any more results to fetch:
```
# First call, everything as expected
Cursor: None, Next cursor: foo, Loaded entities: 10.

# Second call, expected next_page_token = None
Cursor: foo, Next cursor: bar, Loaded entities: 1.

# Third call, again expected next_page_token = None
Cursor: bar, Next cursor: bar, Loaded entities: 0.
```

It seems that `next_page_token` is always not `None` and moreover, if there is not any result to fetch, `next_cursor == cursor`.

Setup:
```
Python 2.7.12
google-cloud-datastore==1.3.0
```

Is this expected? If yes, how can I recognise that there are not more results to fetch? 

I've checked the code briefly and it might be related to https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/datastore/google/cloud/datastore/query.py#L457. It seems like `response_pb.batch.end_cursor == b''` does not hold and thus `next_page_token` is not set to `None`, but not sure why this is the case.



Discussion:
It looks like the API 
returns a separate value "more results" which is not properly handled by the code, see https://cloud.google.com/datastore/docs/reference/rest/v1/projects/runQuery#QueryResultBatch and https://cloud.google.com/datastore/docs/reference/rpc/google.datastore.v1#google.datastore.v1.QueryResultBatch.

Resolving patch:
diff --git a/datastore/google/cloud/datastore/query.py b/datastore/google/cloud/datastore/query.py
--- a/datastore/google/cloud/datastore/query.py
+++ b/datastore/google/cloud/datastore/query.py
@@ -26,9 +26,10 @@
 
 
 _NOT_FINISHED = query_pb2.QueryResultBatch.NOT_FINISHED
+_NO_MORE_RESULTS = query_pb2.QueryResultBatch.NO_MORE_RESULTS
 
 _FINISHED = (
-    query_pb2.QueryResultBatch.NO_MORE_RESULTS,
+    _NO_MORE_RESULTS,
     query_pb2.QueryResultBatch.MORE_RESULTS_AFTER_LIMIT,
     query_pb2.QueryResultBatch.MORE_RESULTS_AFTER_CURSOR,
 )
@@ -470,7 +471,7 @@ def _process_query_results(self, response_pb):
         """
         self._skipped_results = response_pb.batch.skipped_results
 
-        if response_pb.batch.end_cursor == b'':  # Empty-value for bytes.
+        if response_pb.batch.more_results == _NO_MORE_RESULTS:
             self.next_page_token = None
         else:
             self.next_page_token = base64.urlsafe_b64encode(


Test patch:
diff --git a/datastore/tests/unit/test_query.py b/datastore/tests/unit/test_query.py
--- a/datastore/tests/unit/test_query.py
+++ b/datastore/tests/unit/test_query.py
@@ -467,7 +467,7 @@ def test__process_query_results_done(self):
         entity_pbs = [
             _make_entity('World', 1234, 'PROJECT'),
         ]
-        cursor_as_bytes = b''
+        cursor_as_bytes = b'\x9ai\xe7'
         skipped_results = 44
         more_results_enum = query_pb2.QueryResultBatch.NO_MORE_RESULTS
         response_pb = _make_query_response(