Repository: googleapis/python-api-core (python)

Issue:
ServiceUnavailable missing in polling retries
"503 service unavailable" errors are not being retried when long running operations (e.g. [dataproc create_cluster](https://googleapis.dev/python/dataproc/latest/dataproc_v1/services.html?highlight=create_cluster#google.cloud.dataproc_v1.services.cluster_controller.ClusterControllerClient.create_cluster)) are polled.

Currently, the [default retry predicate (retry.py)](https://github.com/googleapis/python-api-core/blob/ed092271ab94c7ef53c40ce8e73ca2c48e97159c/google/api_core/retry.py#L100) includes `google.api_core.exceptions.ServiceUnavailable`, but the [polling retry predicate (polling.py)](https://github.com/googleapis/python-api-core/blob/ed092271ab94c7ef53c40ce8e73ca2c48e97159c/google/api_core/future/polling.py#L32) overrides the former and doesn't include ServiceUnavailable. This doesn't look intentional: if retry.py considers ServiceUnavailable to be transient and retriable, so should polling.py.

This is causing issues for customers: e.g. their Cloud Function crashes during polling when they encounter a single `connection reset`.

Pull request:
feat: add ServiceUnavailable exception to polling retries
Fixes #183 🦕


Patch:
diff --git a/.github/.OwlBot.lock.yaml b/.github/.OwlBot.lock.yaml
index 29084e8..da616c9 100644
--- a/.github/.OwlBot.lock.yaml
+++ b/.github/.OwlBot.lock.yaml
@@ -1,4 +1,3 @@
 docker:
-  digest: sha256:cfc0e802701262c211703c468874d767f65dabe6a1a71d0e07bfc8a3d5175f32
-  image: gcr.io/repo-automation-bots/owlbot-python:latest  
-
+  image: gcr.io/repo-automation-bots/owlbot-python:latest
+  digest: sha256:c66ba3c8d7bc8566f47df841f98cd0097b28fff0b1864c86f5817f4c8c3e8600
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 8912e9b..4f00c7c 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -26,6 +26,6 @@ repos:
     hooks:
     - id: black
 -   repo: https://gitlab.com/pycqa/flake8
-    rev: 3.9.0
+    rev: 3.9.2
     hooks:
     - id: flake8
diff --git a/.repo-metadata.json b/.repo-metadata.json
index 15dd87d..59aa936 100644
--- a/.repo-metadata.json
+++ b/.repo-metadata.json
@@ -4,6 +4,7 @@
   "client_documentation": "https://googleapis.dev/python/google-api-core/latest",
   "release_level": "ga",
   "language": "python",
+  "library_type": "CORE",
   "repo": "googleapis/python-api-core",
   "distribution_name": "google-api-core"
 }
\ No newline at end of file
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 9700ca7..d4e292b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -4,6 +4,45 @@
 
 [1]: https://pypi.org/project/google-api-core/#history
 
+## [1.30.0](https://www.github.com/googleapis/python-api-core/compare/v1.29.0...v1.30.0) (2021-06-08)
+
+
+### Features
+
+* add iterator capability to paged iterators ([#200](https://www.github.com/googleapis/python-api-core/issues/200)) ([3487d68](https://www.github.com/googleapis/python-api-core/commit/3487d68bdab6f20e2ab931c8283f63c94862cf31))
+
+## [1.29.0](https://www.github.com/googleapis/python-api-core/compare/v1.28.0...v1.29.0) (2021-06-02)
+
+
+### Features
+
+* HTTPIterator now accepts a page_size parameter to control page … ([#197](https://www.github.com/googleapis/python-api-core/issues/197)) ([a421913](https://www.github.com/googleapis/python-api-core/commit/a4219137a5bfcf2a6f44780ecdbf475c1129e461))
+
+
+### Documentation
+
+* fix broken links in multiprocessing.rst ([#195](https://www.github.com/googleapis/python-api-core/issues/195)) ([8d8bc51](https://www.github.com/googleapis/python-api-core/commit/8d8bc5150ee5543b4aeb2c271da034a5305d1436))
+
+## [1.28.0](https://www.github.com/googleapis/python-api-core/compare/v1.27.0...v1.28.0) (2021-05-20)
+
+
+### Bug Fixes
+
+* require google-auth>=1.25.0 ([#190](https://www.github.com/googleapis/python-api-core/issues/190)) ([155da5e](https://www.github.com/googleapis/python-api-core/commit/155da5e18cc2fdcfa57de6f956b7d078e79cd4b7))
+
+
+### Miscellaneous Chores
+
+* release 1.28.0 ([#192](https://www.github.com/googleapis/python-api-core/issues/192)) ([11b5da4](https://www.github.com/googleapis/python-api-core/commit/11b5da426a842541ca2b861d3387fc312b3f5b60))
+
+## [1.27.0](https://www.github.com/googleapis/python-api-core/compare/v1.26.3...v1.27.0) (2021-05-18)
+
+
+### Features
+
+* Add support for `rest/` token in `x-goog-api-client` header ([#189](https://www.github.com/googleapis/python-api-core/issues/189)) ([15aca6b](https://www.github.com/googleapis/python-api-core/commit/15aca6b288b2ec5ce0251e442e1dfa7f52e1b124))
+* retry google.auth TransportError and requests ConnectionError ([#178](https://www.github.com/googleapis/python-api-core/issues/178)) ([6ae04a8](https://www.github.com/googleapis/python-api-core/commit/6ae04a8d134fffe13f06081e15f9723c1b2ea334))
+
 ### [1.26.3](https://www.github.com/googleapis/python-api-core/compare/v1.26.2...v1.26.3) (2021-03-25)
 
 
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
index 1116150..03c4abe 100644
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -160,21 +160,7 @@ Running System Tests
   auth settings and change some configuration in your project to
   run all the tests.
 
-- System tests will be run against an actual project and
-  so you'll need to provide some environment variables to facilitate
-  authentication to your project:
-
-  - ``GOOGLE_APPLICATION_CREDENTIALS``: The path to a JSON key file;
-    Such a file can be downloaded directly from the developer's console by clicking
-    "Generate new JSON key". See private key
-    `docs <https://cloud.google.com/storage/docs/authentication#generating-a-private-key>`__
-    for more details.
-
-- Once you have downloaded your json keys, set the environment variable 
-  ``GOOGLE_APPLICATION_CREDENTIALS`` to the absolute path of the json file::
-
-   $ export GOOGLE_APPLICATION_CREDENTIALS="/Users/<your_username>/path/to/app_credentials.json"
-
+- System tests will be run against an actual project. You should use local credentials from gcloud when possible. See `Best practices for application authentication <https://cloud.google.com/docs/authentication/best-practices-applications#local_development_and_testing_with_the>`__. Some tests require a service account. For those tests see `Authenticating as a service account <https://cloud.google.com/docs/authentication/production>`__.
 
 *************
 Test Coverage
diff --git a/SECURITY.md b/SECURITY.md
new file mode 100644
index 0000000..8b58ae9
--- /dev/null
+++ b/SECURITY.md
@@ -0,0 +1,7 @@
+# Security Policy
+
+To report a security issue, please use [g.co/vulnz](https://g.co/vulnz).
+
+The Google Security Team will respond within 5 working days of your report on g.co/vulnz.
+
+We use g.co/vulnz for our intake, and do coordination and disclosure here using GitHub Security Advisory to privately discuss and fix the issue.
diff --git a/docs/conf.py b/docs/conf.py
index 9ec0452..1e73c44 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -366,6 +366,7 @@ intersphinx_mapping = {
     ),
     "grpc": ("https://grpc.github.io/grpc/python/", None),
     "proto-plus": ("https://proto-plus-python.readthedocs.io/en/latest/", None),
+    "protobuf": ("https://googleapis.dev/python/protobuf/latest/", None),
     
 }
 
diff --git a/docs/multiprocessing.rst b/docs/multiprocessing.rst
index 1cb29d4..536d17b 100644
--- a/docs/multiprocessing.rst
+++ b/docs/multiprocessing.rst
@@ -1,7 +1,7 @@
 .. note::
 
-   Because this client uses :mod:`grpcio` library, it is safe to
+   Because this client uses :mod:`grpc` library, it is safe to
    share instances across threads. In multiprocessing scenarios, the best
    practice is to create client instances *after* the invocation of
-   :func:`os.fork` by :class:`multiprocessing.Pool` or
+   :func:`os.fork` by :class:`multiprocessing.pool.Pool` or
    :class:`multiprocessing.Process`.
diff --git a/google/api_core/client_info.py b/google/api_core/client_info.py
index 6c04d5d..adca5f3 100644
--- a/google/api_core/client_info.py
+++ b/google/api_core/client_info.py
@@ -54,6 +54,7 @@ class ClientInfo(object):
         user_agent (Optional[str]): Prefix to the user agent header. This is
             used to supply information such as application name or partner tool.
             Recommended format: ``application-or-tool-ID/major.minor.version``.
+        rest_version (Optional[str]): The requests library version.
     """
 
     def __init__(
@@ -64,6 +65,7 @@ class ClientInfo(object):
         gapic_version=None,
         client_library_version=None,
         user_agent=None,
+        rest_version=None,
     ):
         self.python_version = python_version
         self.grpc_version = grpc_version
@@ -71,6 +73,7 @@ class ClientInfo(object):
         self.gapic_version = gapic_version
         self.client_library_version = client_library_version
         self.user_agent = user_agent
+        self.rest_version = rest_version
 
     def to_user_agent(self):
         """Returns the user-agent string for this client info."""
@@ -87,6 +90,9 @@ class ClientInfo(object):
         if self.grpc_version is not None:
             ua += "grpc/{grpc_version} "
 
+        if self.rest_version is not None:
+            ua += "rest/{rest_version} "
+
         ua += "gax/{api_core_version} "
 
         if self.gapic_version is not None:
diff --git a/google/api_core/future/polling.py b/google/api_core/future/polling.py
index 6466838..2f80efb 100644
--- a/google/api_core/future/polling.py
+++ b/google/api_core/future/polling.py
@@ -34,6 +34,7 @@ RETRY_PREDICATE = retry.if_exception_type(
     exceptions.TooManyRequests,
     exceptions.InternalServerError,
     exceptions.BadGateway,
+    exceptions.ServiceUnavailable,
 )
 DEFAULT_RETRY = retry.Retry(predicate=RETRY_PREDICATE)
 
diff --git a/google/api_core/grpc_helpers.py b/google/api_core/grpc_helpers.py
index 5937f18..e8f9ee8 100644
--- a/google/api_core/grpc_helpers.py
+++ b/google/api_core/grpc_helpers.py
@@ -17,7 +17,6 @@
 import collections
 
 import grpc
-from packaging import version
 import pkg_resources
 import six
 
@@ -44,11 +43,6 @@ except AttributeError:
     except pkg_resources.DistributionNotFound:  # pragma: NO COVER
         _GOOGLE_AUTH_VERSION = None
 
-if _GOOGLE_AUTH_VERSION is not None and version.parse(_GOOGLE_AUTH_VERSION) >= version.parse("1.25.0"):
-    _GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST = True
-else:
-    _GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST = False
-
 # The list of gRPC Callable interfaces that return iterators.
 _STREAM_WRAP_CLASSES = (grpc.UnaryStreamMultiCallable, grpc.StreamStreamMultiCallable)
 
@@ -232,38 +226,19 @@ def _create_composite_credentials(
         )
 
     if credentials_file:
-        # TODO: remove this if/else once google-auth >= 1.25.0 is required
-        if _GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-            credentials, _ = google.auth.load_credentials_from_file(
-                credentials_file,
-                scopes=scopes,
-                default_scopes=default_scopes
-            )
-        else:
-            credentials, _ = google.auth.load_credentials_from_file(
-                credentials_file,
-                scopes=scopes or default_scopes,
-            )
+        credentials, _ = google.auth.load_credentials_from_file(
+            credentials_file,
+            scopes=scopes,
+            default_scopes=default_scopes
+        )
     elif credentials:
-        # TODO: remove this if/else once google-auth >= 1.25.0 is required
-        if _GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-            credentials = google.auth.credentials.with_scopes_if_required(
-                credentials,
-                scopes=scopes,
-                default_scopes=default_scopes
-            )
-        else:
-            credentials = google.auth.credentials.with_scopes_if_required(
-                credentials,
-                scopes=scopes or default_scopes,
-            )
-
+        credentials = google.auth.credentials.with_scopes_if_required(
+            credentials,
+            scopes=scopes,
+            default_scopes=default_scopes
+        )
     else:
-        # TODO: remove this if/else once google-auth >= 1.25.0 is required
-        if _GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-            credentials, _ = google.auth.default(scopes=scopes, default_scopes=default_scopes)
-        else:
-            credentials, _ = google.auth.default(scopes=scopes or default_scopes)
+        credentials, _ = google.auth.default(scopes=scopes, default_scopes=default_scopes)
 
     if quota_project_id and isinstance(credentials, google.auth.credentials.CredentialsWithQuotaProject):
         credentials = credentials.with_quota_project(quota_project_id)
@@ -271,16 +246,9 @@ def _create_composite_credentials(
     request = google.auth.transport.requests.Request()
 
     # Create the metadata plugin for inserting the authorization header.
-
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if _GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        metadata_plugin = google.auth.transport.grpc.AuthMetadataPlugin(
-            credentials, request, default_host=default_host,
-        )
-    else:
-        metadata_plugin = google.auth.transport.grpc.AuthMetadataPlugin(
-            credentials, request
-        )
+    metadata_plugin = google.auth.transport.grpc.AuthMetadataPlugin(
+        credentials, request, default_host=default_host,
+    )
 
     # Create a set of grpc.CallCredentials using the metadata plugin.
     google_auth_credentials = grpc.metadata_call_credentials(metadata_plugin)
diff --git a/google/api_core/page_iterator.py b/google/api_core/page_iterator.py
index 11a92d3..49879bc 100644
--- a/google/api_core/page_iterator.py
+++ b/google/api_core/page_iterator.py
@@ -170,6 +170,8 @@ class Iterator(object):
         max_results=None,
     ):
         self._started = False
+        self.__active_iterator = None
+
         self.client = client
         """Optional[Any]: The client that created this iterator."""
         self.item_to_value = item_to_value
@@ -179,7 +181,7 @@ class Iterator(object):
             single item.
         """
         self.max_results = max_results
-        """int: The maximum number of results to fetch."""
+        """int: The maximum number of results to fetch"""
 
         # The attributes below will change over the life of the iterator.
         self.page_number = 0
@@ -228,6 +230,14 @@ class Iterator(object):
         self._started = True
         return self._items_iter()
 
+    def __next__(self):
+        if self.__active_iterator is None:
+            self.__active_iterator = iter(self)
+        return next(self.__active_iterator)
+
+    # Preserve Python 2 compatibility.
+    next = __next__
+
     def _page_iter(self, increment):
         """Generator of pages of API responses.
 
@@ -298,7 +308,8 @@ class HTTPIterator(Iterator):
             can be found.
         page_token (str): A token identifying a page in a result set to start
             fetching results from.
-        max_results (int): The maximum number of results to fetch.
+        page_size (int): The maximum number of results to fetch per page
+        max_results (int): The maximum number of results to fetch
         extra_params (dict): Extra query string parameters for the
             API call.
         page_start (Callable[
@@ -329,6 +340,7 @@ class HTTPIterator(Iterator):
         item_to_value,
         items_key=_DEFAULT_ITEMS_KEY,
         page_token=None,
+        page_size=None,
         max_results=None,
         extra_params=None,
         page_start=_do_nothing_page_start,
@@ -341,6 +353,7 @@ class HTTPIterator(Iterator):
         self.path = path
         self._items_key = items_key
         self.extra_params = extra_params
+        self._page_size = page_size
         self._page_start = page_start
         self._next_token = next_token
         # Verify inputs / provide defaults.
@@ -399,8 +412,18 @@ class HTTPIterator(Iterator):
         result = {}
         if self.next_page_token is not None:
             result[self._PAGE_TOKEN] = self.next_page_token
+
+        page_size = None
         if self.max_results is not None:
-            result[self._MAX_RESULTS] = self.max_results - self.num_results
+            page_size = self.max_results - self.num_results
+            if self._page_size is not None:
+                page_size = min(page_size, self._page_size)
+        elif self._page_size is not None:
+            page_size = self._page_size
+
+        if page_size is not None:
+            result[self._MAX_RESULTS] = page_size
+
         result.update(self.extra_params)
         return result
 
diff --git a/google/api_core/page_iterator_async.py b/google/api_core/page_iterator_async.py
index a0aa41a..c072575 100644
--- a/google/api_core/page_iterator_async.py
+++ b/google/api_core/page_iterator_async.py
@@ -101,6 +101,8 @@ class AsyncIterator(abc.ABC):
         max_results=None,
     ):
         self._started = False
+        self.__active_aiterator = None
+
         self.client = client
         """Optional[Any]: The client that created this iterator."""
         self.item_to_value = item_to_value
@@ -159,6 +161,11 @@ class AsyncIterator(abc.ABC):
         self._started = True
         return self._items_aiter()
 
+    async def __anext__(self):
+        if self.__active_aiterator is None:
+            self.__active_aiterator = self.__aiter__()
+        return await self.__active_aiterator.__anext__()
+
     async def _page_aiter(self, increment):
         """Generator of pages of API responses.
 
diff --git a/google/api_core/version.py b/google/api_core/version.py
index 829945c..ed763cf 100644
--- a/google/api_core/version.py
+++ b/google/api_core/version.py
@@ -12,4 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-__version__ = "1.26.3"
+__version__ = "1.30.0"
diff --git a/setup.py b/setup.py
index 5de5aaf..26fab7e 100644
--- a/setup.py
+++ b/setup.py
@@ -31,7 +31,7 @@ release_status = "Development Status :: 5 - Production/Stable"
 dependencies = [
     "googleapis-common-protos >= 1.6.0, < 2.0dev",
     "protobuf >= 3.12.0",
-    "google-auth >= 1.21.1, < 2.0dev",
+    "google-auth >= 1.25.0, < 2.0dev",
     "requests >= 2.18.0, < 3.0.0dev",
     "setuptools >= 40.3.0",
     "packaging >= 14.3",


Test patch:
diff --git a/testing/constraints-3.6.txt b/testing/constraints-3.6.txt
index 1fcd193..ff5b4a7 100644
--- a/testing/constraints-3.6.txt
+++ b/testing/constraints-3.6.txt
@@ -7,7 +7,7 @@
 # Then this file should have foo==1.14.0
 googleapis-common-protos==1.6.0
 protobuf==3.12.0
-google-auth==1.21.1
+google-auth==1.25.0
 requests==2.18.0
 setuptools==40.3.0
 packaging==14.3
diff --git a/tests/asyncio/test_grpc_helpers_async.py b/tests/asyncio/test_grpc_helpers_async.py
index 3461cbe..868018c 100644
--- a/tests/asyncio/test_grpc_helpers_async.py
+++ b/tests/asyncio/test_grpc_helpers_async.py
@@ -18,7 +18,6 @@ import mock
 import pytest
 
 from google.api_core import exceptions
-from google.api_core import grpc_helpers
 from google.api_core import grpc_helpers_async
 import google.auth.credentials
 
@@ -276,11 +275,7 @@ def test_create_channel_implicit(grpc_secure_channel, default, composite_creds_c
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=None)
-    else:
-        default.assert_called_once_with(scopes=None)
+    default.assert_called_once_with(scopes=None, default_scopes=None)
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
 
@@ -306,14 +301,8 @@ def test_create_channel_implicit_with_default_host(grpc_secure_channel, default,
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=None)
-        auth_metadata_plugin.assert_called_once_with(mock.sentinel.credentials, mock.sentinel.Request, default_host=default_host)
-    else:
-        default.assert_called_once_with(scopes=None)
-        auth_metadata_plugin.assert_called_once_with(mock.sentinel.credentials, mock.sentinel.Request)
-
+    default.assert_called_once_with(scopes=None, default_scopes=None)
+    auth_metadata_plugin.assert_called_once_with(mock.sentinel.credentials, mock.sentinel.Request, default_host=default_host)
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
 
@@ -332,12 +321,7 @@ def test_create_channel_implicit_with_ssl_creds(
 
     grpc_helpers_async.create_channel(target, ssl_credentials=ssl_creds)
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=None)
-    else:
-        default.assert_called_once_with(scopes=None)
-
+    default.assert_called_once_with(scopes=None, default_scopes=None)
     composite_creds_call.assert_called_once_with(ssl_creds, mock.ANY)
     composite_creds = composite_creds_call.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
@@ -360,12 +344,7 @@ def test_create_channel_implicit_with_scopes(
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=["one", "two"], default_scopes=None)
-    else:
-        default.assert_called_once_with(scopes=["one", "two"])
-
+    default.assert_called_once_with(scopes=["one", "two"], default_scopes=None)
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
 
@@ -386,12 +365,7 @@ def test_create_channel_implicit_with_default_scopes(
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=["three", "four"])
-    else:
-        default.assert_called_once_with(scopes=["three", "four"])
-
+    default.assert_called_once_with(scopes=None, default_scopes=["three", "four"])
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
 
@@ -417,12 +391,7 @@ def test_create_channel_explicit(grpc_secure_channel, auth_creds, composite_cred
 
     channel = grpc_helpers_async.create_channel(target, credentials=mock.sentinel.credentials)
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        auth_creds.assert_called_once_with(mock.sentinel.credentials, scopes=None, default_scopes=None)
-    else:
-        auth_creds.assert_called_once_with(mock.sentinel.credentials, scopes=None)
-
+    auth_creds.assert_called_once_with(mock.sentinel.credentials, scopes=None, default_scopes=None)
     assert channel is grpc_secure_channel.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
@@ -440,12 +409,8 @@ def test_create_channel_explicit_scoped(grpc_secure_channel, composite_creds_cal
     channel = grpc_helpers_async.create_channel(
         target, credentials=credentials, scopes=scopes
     )
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes)
 
+    credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
     assert channel is grpc_secure_channel.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
@@ -464,12 +429,7 @@ def test_create_channel_explicit_default_scopes(grpc_secure_channel, composite_c
         target, credentials=credentials, default_scopes=default_scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes=None, default_scopes=default_scopes)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes=default_scopes)
-
+    credentials.with_scopes.assert_called_once_with(scopes=None, default_scopes=default_scopes)
     assert channel is grpc_secure_channel.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
@@ -508,12 +468,7 @@ def test_create_channnel_with_credentials_file(load_credentials_from_file, grpc_
         target, credentials_file=credentials_file
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=None)
-    else:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None)
-
+    google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=None)
     assert channel is grpc_secure_channel.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
@@ -536,12 +491,7 @@ def test_create_channel_with_credentials_file_and_scopes(load_credentials_from_f
         target, credentials_file=credentials_file, scopes=scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=scopes, default_scopes=None)
-    else:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=scopes)
-
+    google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=scopes, default_scopes=None)
     assert channel is grpc_secure_channel.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
@@ -564,12 +514,7 @@ def test_create_channel_with_credentials_file_and_default_scopes(load_credential
         target, credentials_file=credentials_file, default_scopes=default_scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=default_scopes)
-    else:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=default_scopes)
-
+    google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=default_scopes)
     assert channel is grpc_secure_channel.return_value
     grpc_secure_channel.assert_called_once_with(target, composite_creds)
 
@@ -585,12 +530,7 @@ def test_create_channel_without_grpc_gcp(grpc_secure_channel):
 
     grpc_helpers_async.create_channel(target, credentials=credentials, scopes=scopes)
     grpc_secure_channel.assert_called()
-
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes)
+    credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
 
 
 @pytest.mark.asyncio
diff --git a/tests/asyncio/test_page_iterator_async.py b/tests/asyncio/test_page_iterator_async.py
index 42fac2a..4abacc6 100644
--- a/tests/asyncio/test_page_iterator_async.py
+++ b/tests/asyncio/test_page_iterator_async.py
@@ -47,6 +47,33 @@ class TestAsyncIterator:
         assert iterator.next_page_token == token
         assert iterator.num_results == 0
 
+    @pytest.mark.asyncio
+    async def test_anext(self):
+        parent = mock.sentinel.parent
+        page_1 = page_iterator_async.Page(
+            parent, ("item 1.1", "item 1.2"), page_iterator_async._item_to_value_identity
+        )
+        page_2 = page_iterator_async.Page(
+            parent, ("item 2.1",), page_iterator_async._item_to_value_identity
+        )
+
+        async_iterator = PageAsyncIteratorImpl(None, None)
+        async_iterator._next_page = mock.AsyncMock(side_effect=[page_1, page_2, None])
+
+        # Consume items and check the state of the async_iterator.
+        assert async_iterator.num_results == 0
+        assert await async_iterator.__anext__() == "item 1.1"
+        assert async_iterator.num_results == 1
+
+        assert await async_iterator.__anext__() == "item 1.2"
+        assert async_iterator.num_results == 2
+
+        assert await async_iterator.__anext__() == "item 2.1"
+        assert async_iterator.num_results == 3
+
+        with pytest.raises(StopAsyncIteration):
+            await async_iterator.__anext__()
+
     def test_pages_property_starts(self):
         iterator = PageAsyncIteratorImpl(None, None)
 
diff --git a/tests/unit/test_client_info.py b/tests/unit/test_client_info.py
index 0eb17c5..f2274ec 100644
--- a/tests/unit/test_client_info.py
+++ b/tests/unit/test_client_info.py
@@ -24,6 +24,7 @@ def test_constructor_defaults():
     assert info.api_core_version is not None
     assert info.gapic_version is None
     assert info.client_library_version is None
+    assert info.rest_version is None
 
 
 def test_constructor_options():
@@ -33,7 +34,8 @@ def test_constructor_options():
         api_core_version="3",
         gapic_version="4",
         client_library_version="5",
-        user_agent="6"
+        user_agent="6",
+        rest_version="7",
     )
 
     assert info.python_version == "1"
@@ -42,6 +44,7 @@ def test_constructor_options():
     assert info.gapic_version == "4"
     assert info.client_library_version == "5"
     assert info.user_agent == "6"
+    assert info.rest_version == "7"
 
 
 def test_to_user_agent_minimal():
@@ -67,3 +70,19 @@ def test_to_user_agent_full():
     user_agent = info.to_user_agent()
 
     assert user_agent == "app-name/1.0 gl-python/1 grpc/2 gax/3 gapic/4 gccl/5"
+
+
+def test_to_user_agent_rest():
+    info = client_info.ClientInfo(
+        python_version="1",
+        grpc_version=None,
+        rest_version="2",
+        api_core_version="3",
+        gapic_version="4",
+        client_library_version="5",
+        user_agent="app-name/1.0",
+    )
+
+    user_agent = info.to_user_agent()
+
+    assert user_agent == "app-name/1.0 gl-python/1 rest/2 gax/3 gapic/4 gccl/5"
diff --git a/tests/unit/test_grpc_helpers.py b/tests/unit/test_grpc_helpers.py
index 4e0ab80..8c6202b 100644
--- a/tests/unit/test_grpc_helpers.py
+++ b/tests/unit/test_grpc_helpers.py
@@ -234,11 +234,7 @@ def test_create_channel_implicit(grpc_secure_channel, default, composite_creds_c
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=None)
-    else:
-        default.assert_called_once_with(scopes=None)
+    default.assert_called_once_with(scopes=None, default_scopes=None)
 
     if grpc_helpers.HAS_GRPC_GCP:
         grpc_secure_channel.assert_called_once_with(target, composite_creds, None)
@@ -268,15 +264,8 @@ def test_create_channel_implicit_with_default_host(grpc_secure_channel, default,
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    print(grpc_helpers._GOOGLE_AUTH_VERSION)
-    print(grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST)
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=None)
-        auth_metadata_plugin.assert_called_once_with(mock.sentinel.credentials, mock.sentinel.Request, default_host=default_host)
-    else:
-        default.assert_called_once_with(scopes=None)
-        auth_metadata_plugin.assert_called_once_with(mock.sentinel.credentials, mock.sentinel.Request)
+    default.assert_called_once_with(scopes=None, default_scopes=None)
+    auth_metadata_plugin.assert_called_once_with(mock.sentinel.credentials, mock.sentinel.Request, default_host=default_host)
 
     if grpc_helpers.HAS_GRPC_GCP:
         grpc_secure_channel.assert_called_once_with(target, composite_creds, None)
@@ -300,11 +289,7 @@ def test_create_channel_implicit_with_ssl_creds(
 
     grpc_helpers.create_channel(target, ssl_credentials=ssl_creds)
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=None)
-    else:
-        default.assert_called_once_with(scopes=None)
+    default.assert_called_once_with(scopes=None, default_scopes=None)
 
     composite_creds_call.assert_called_once_with(ssl_creds, mock.ANY)
     composite_creds = composite_creds_call.return_value
@@ -331,11 +316,7 @@ def test_create_channel_implicit_with_scopes(
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=["one", "two"], default_scopes=None)
-    else:
-        default.assert_called_once_with(scopes=["one", "two"])
+    default.assert_called_once_with(scopes=["one", "two"], default_scopes=None)
 
     if grpc_helpers.HAS_GRPC_GCP:
         grpc_secure_channel.assert_called_once_with(target, composite_creds, None)
@@ -360,11 +341,7 @@ def test_create_channel_implicit_with_default_scopes(
 
     assert channel is grpc_secure_channel.return_value
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        default.assert_called_once_with(scopes=None, default_scopes=["three", "four"])
-    else:
-        default.assert_called_once_with(scopes=["three", "four"])
+    default.assert_called_once_with(scopes=None, default_scopes=["three", "four"])
 
     if grpc_helpers.HAS_GRPC_GCP:
         grpc_secure_channel.assert_called_once_with(target, composite_creds, None)
@@ -392,11 +369,8 @@ def test_create_channel_explicit(grpc_secure_channel, auth_creds, composite_cred
 
     channel = grpc_helpers.create_channel(target, credentials=mock.sentinel.credentials)
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        auth_creds.assert_called_once_with(mock.sentinel.credentials, scopes=None, default_scopes=None)
-    else:
-        auth_creds.assert_called_once_with(mock.sentinel.credentials, scopes=None)
+    auth_creds.assert_called_once_with(mock.sentinel.credentials, scopes=None, default_scopes=None)
+
     assert channel is grpc_secure_channel.return_value
     if grpc_helpers.HAS_GRPC_GCP:
         grpc_secure_channel.assert_called_once_with(target, composite_creds, None)
@@ -418,11 +392,7 @@ def test_create_channel_explicit_scoped(grpc_secure_channel, composite_creds_cal
         target, credentials=credentials, scopes=scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes)
+    credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
 
     assert channel is grpc_secure_channel.return_value
     if grpc_helpers.HAS_GRPC_GCP:
@@ -445,11 +415,7 @@ def test_create_channel_explicit_default_scopes(grpc_secure_channel, composite_c
         target, credentials=credentials, default_scopes=default_scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes=None, default_scopes=default_scopes)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes=default_scopes)
+    credentials.with_scopes.assert_called_once_with(scopes=None, default_scopes=default_scopes)
 
     assert channel is grpc_secure_channel.return_value
     if grpc_helpers.HAS_GRPC_GCP:
@@ -498,11 +464,7 @@ def test_create_channel_with_credentials_file(load_credentials_from_file, grpc_s
         target, credentials_file=credentials_file
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=None)
-    else:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None)
+    google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=None)
 
     assert channel is grpc_secure_channel.return_value
     if grpc_helpers.HAS_GRPC_GCP:
@@ -529,11 +491,7 @@ def test_create_channel_with_credentials_file_and_scopes(load_credentials_from_f
         target, credentials_file=credentials_file, scopes=scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=scopes, default_scopes=None)
-    else:
-        google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=scopes)
+    google.auth.load_credentials_from_file.assert_called_once_with(credentials_file, scopes=scopes, default_scopes=None)
 
     assert channel is grpc_secure_channel.return_value
     if grpc_helpers.HAS_GRPC_GCP:
@@ -560,11 +518,7 @@ def test_create_channel_with_credentials_file_and_default_scopes(load_credential
         target, credentials_file=credentials_file, default_scopes=default_scopes
     )
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=default_scopes)
-    else:
-        load_credentials_from_file.assert_called_once_with(credentials_file, scopes=default_scopes)
+    load_credentials_from_file.assert_called_once_with(credentials_file, scopes=None, default_scopes=default_scopes)
 
     assert channel is grpc_secure_channel.return_value
     if grpc_helpers.HAS_GRPC_GCP:
@@ -587,11 +541,7 @@ def test_create_channel_with_grpc_gcp(grpc_gcp_secure_channel):
     grpc_helpers.create_channel(target, credentials=credentials, scopes=scopes)
     grpc_gcp_secure_channel.assert_called()
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes)
+    credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
 
 
 @pytest.mark.skipif(grpc_helpers.HAS_GRPC_GCP, reason="grpc_gcp module not available")
@@ -606,11 +556,7 @@ def test_create_channel_without_grpc_gcp(grpc_secure_channel):
     grpc_helpers.create_channel(target, credentials=credentials, scopes=scopes)
     grpc_secure_channel.assert_called()
 
-    # TODO: remove this if/else once google-auth >= 1.25.0 is required
-    if grpc_helpers._GOOGLE_AUTH_HAS_DEFAULT_SCOPES_AND_DEFAULT_HOST:
-        credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
-    else:
-        credentials.with_scopes.assert_called_once_with(scopes)
+    credentials.with_scopes.assert_called_once_with(scopes, default_scopes=None)
 
 
 class TestChannelStub(object):
diff --git a/tests/unit/test_page_iterator.py b/tests/unit/test_page_iterator.py
index 2bf7424..97b0657 100644
--- a/tests/unit/test_page_iterator.py
+++ b/tests/unit/test_page_iterator.py
@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import math
 import types
 
 import mock
@@ -108,6 +109,26 @@ class TestIterator(object):
         assert iterator.next_page_token == token
         assert iterator.num_results == 0
 
+    def test_next(self):
+        iterator = PageIteratorImpl(None, None)
+        page_1 = page_iterator.Page(
+            iterator, ("item 1.1", "item 1.2"), page_iterator._item_to_value_identity
+        )
+        page_2 = page_iterator.Page(
+            iterator, ("item 2.1",), page_iterator._item_to_value_identity
+        )
+        iterator._next_page = mock.Mock(side_effect=[page_1, page_2, None])
+
+        result = next(iterator)
+        assert result == "item 1.1"
+        result = next(iterator)
+        assert result == "item 1.2"
+        result = next(iterator)
+        assert result == "item 2.1"
+
+        with pytest.raises(StopIteration):
+            next(iterator)
+
     def test_pages_property_starts(self):
         iterator = PageIteratorImpl(None, None)
 
@@ -235,6 +256,7 @@ class TestHTTPIterator(object):
         assert iterator.page_number == 0
         assert iterator.next_page_token is None
         assert iterator.num_results == 0
+        assert iterator._page_size is None
 
     def test_constructor_w_extra_param_collision(self):
         extra_params = {"pageToken": "val"}
@@ -432,6 +454,66 @@ class TestHTTPIterator(object):
         with pytest.raises(ValueError):
             iterator._get_next_page_response()
 
+    @pytest.mark.parametrize(
+        "page_size,max_results,pages",
+        [(3, None, False), (3, 8, False), (3, None, True), (3, 8, True)])
+    def test_page_size_items(self, page_size, max_results, pages):
+        path = "/foo"
+        NITEMS = 10
+
+        n = [0]  # blast you python 2!
+
+        def api_request(*args, **kw):
+            assert not args
+            query_params = dict(
+                maxResults=(
+                    page_size if max_results is None
+                    else min(page_size, max_results - n[0]))
+            )
+            if n[0]:
+                query_params.update(pageToken='test')
+            assert kw == {'method': 'GET', 'path': '/foo',
+                          'query_params': query_params}
+            n_items = min(kw['query_params']['maxResults'], NITEMS - n[0])
+            items = [dict(name=str(i + n[0])) for i in range(n_items)]
+            n[0] += n_items
+            result = dict(items=items)
+            if n[0] < NITEMS:
+                result.update(nextPageToken='test')
+            return result
+
+        iterator = page_iterator.HTTPIterator(
+            mock.sentinel.client,
+            api_request,
+            path=path,
+            item_to_value=page_iterator._item_to_value_identity,
+            page_size=page_size,
+            max_results=max_results,
+        )
+
+        assert iterator.num_results == 0
+
+        n_results = max_results if max_results is not None else NITEMS
+        if pages:
+            items_iter = iter(iterator.pages)
+            npages = int(math.ceil(float(n_results) / page_size))
+            for ipage in range(npages):
+                assert (
+                    list(six.next(items_iter)) == [
+                        dict(name=str(i))
+                        for i in range(ipage * page_size,
+                                       min((ipage + 1) * page_size, n_results),
+                                       )
+                    ])
+        else:
+            items_iter = iter(iterator)
+            for i in range(n_results):
+                assert six.next(items_iter) == dict(name=str(i))
+                assert iterator.num_results == i + 1
+
+        with pytest.raises(StopIteration):
+            six.next(items_iter)
+
 
 class TestGRPCIterator(object):
     def test_constructor(self):