Repository: googleapis/google-cloud-python

Issue:
Pubsub API redesign broke support for PUBSUB_EMULATOR_HOST
Broke 2 days ago with this diff:
https://github.com/GoogleCloudPlatform/google-cloud-python/commit/4a8e155c0e85bf80edbf2e5f1e9342bc3f571acb#diff-c15e14c96ae7c1e14ebdff735c3080e6L62


Discussion:
Hi @bpicolo,
Thanks for letting me know.

I admit unfamiliarity with the Pub/Sub emulator. I will look into what I need to do to make this work for you. If it is just an alternative hostname, that _will_ work, but the incantation is slightly different.
@lukesneeringer Let's chat about this. I **had** great support for Pub/Sub, Bigtable and Datastore, but it got nuked when we switched from `tox` to `nox`.
@lukesneeringer  Should just be the hostname I would think (maybe https cert checking and the like as well, unsure what the actual code path looks like)
Seems worthwhile to add some sort of acceptance tests for this?
@bpicolo Probably.:-)

If it is _just_ hostname, you can send the [`service_path`](https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/pubsub/google/cloud/gapic/pubsub/v1/publisher_client.py#L126) keyword argument to the client (it gets passed to that object if added in `**kwargs` [here](https://github.com/GoogleCloudPlatform/google-cloud-python/blob/master/pubsub/google/cloud/pubsub_v1/publisher/client.py#L60).
We previously **did not** have any CI testing for the emulator integration, but now that we control the Docker image for CircleCI, it would be **very** easy to add the `gcloud` SDK (where the emulators live) to the Docker image. Yay!
@lukesneeringer & @dhermes Is there an ETA for a fix? The workaround with the **kwargs doesn't work since there's still some https stuff failing alongside it...
Same problem here, hoping for a quick fix as we're using the emulator to test most of our services..
:-(  uhhh, how were the API changes tested before the release if the emulator broke  #test-in-prod
(This issue is a blocker for me deploying this wonderful new client 😢)

@dhermes 

> @lukesneeringer Let's chat about this. I had great support for Pub/Sub, Bigtable and Datastore, but it got nuked when we switched from tox to nox.

Thank you for your work on emulator support, however for Pub/Sub there was a noticeable drawback, which was that the client library required _any_ valid GCP service account credentials before it would proceed to use the emulator. Hopefully the replaced emulator support won't have this drawback and will use the emulator without credentials, as e.g. the Datastore client does.
Hi team, is this issue on the work schedule at all? This part of our system needs some refactoring and I'm trying to work out if we should delay so we can upgrade to the new client. Thanks.
Any workaround here? Passing in the service_path and service_port to the client doesn't seem to work and prints the following error:

`E1025 00:04:54.085080000 140737010254784 ssl_transport_security.c:940] Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.`
Hi,

According to Google documentation the developer (users of the library) should not have to do anything in code to use the emulator. Only set the proper environment variable. Also that is the way it worked before this ticket (apparently).

There are two things missing IMHO: 1) a test case that checks that when the PUBSUB_EMULATOR_HOST is set all requests to PUBSUB are forwarded there, and 2) support inside both the subscriber and the publisher to set service_path and service_port accordingly at runtime without user intervention.

Also as per @suhaasprasad comment, the client should allow for non-SSL connections when honouring the EMULATOR variable.

Am I missing something?
Temporary workaround (ugly) solves the SSL problem:
```python
        if os.environ.get('PUBSUB_EMULATOR_HOST'):
            grpc_channel = grpc.insecure_channel(os.environ['PUBSUB_EMULATOR_HOST'])
            publisher = pubsub_v1.PublisherClient(channel=grpc_channel)
        else:
            publisher = pubsub_v1.PublisherClient()
```
I'm still unclear as to the root issue, but I've reverted to using version 0.27.0 of the google-cloud-pubsub library for now.

Resolving patch:
diff --git a/pubsub/google/cloud/pubsub_v1/publisher/client.py b/pubsub/google/cloud/pubsub_v1/publisher/client.py
--- a/pubsub/google/cloud/pubsub_v1/publisher/client.py
+++ b/pubsub/google/cloud/pubsub_v1/publisher/client.py
@@ -15,11 +15,14 @@
 from __future__ import absolute_import
 
 import copy
+import os
 import pkg_resources
 import threading
 
+import grpc
 import six
 
+from google.api_core import grpc_helpers
 from google.cloud.gapic.pubsub.v1 import publisher_client
 
 from google.cloud.pubsub_v1 import _gapic
@@ -53,6 +56,28 @@ class Client(object):
             Generally, you should not need to set additional keyword arguments.
     """
     def __init__(self, batch_settings=(), batch_class=thread.Batch, **kwargs):
+        # Sanity check: Is our goal to use the emulator?
+        # If so, create a grpc insecure channel with the emulator host
+        # as the target.
+        if os.environ.get('PUBSUB_EMULATOR_HOST'):
+            kwargs['channel'] = grpc.insecure_channel(
+                target=os.environ.get('PUBSUB_EMULATOR_HOST'),
+            )
+
+        # Use a custom channel.
+        # We need this in order to set appropriate default message size and
+        # keepalive options.
+        if 'channel' not in kwargs:
+            kwargs['channel'] = grpc_helpers.create_channel(
+                credentials=kwargs.get('credentials', None),
+                target=self.target,
+                scopes=publisher_client.PublisherClient._ALL_SCOPES,
+                options={
+                    'grpc.max_send_message_length': -1,
+                    'grpc.max_receive_message_length': -1,
+                }.items(),
+            )
+
         # Add the metrics headers, and instantiate the underlying GAPIC
         # client.
         kwargs['lib_name'] = 'gccl'
@@ -66,6 +91,18 @@ def __init__(self, batch_settings=(), batch_class=thread.Batch, **kwargs):
         self._batch_lock = threading.Lock()
         self._batches = {}
 
+    @property
+    def target(self):
+        """Return the target (where the API is).
+
+        Returns:
+            str: The location of the API.
+        """
+        return '{host}:{port}'.format(
+            host=publisher_client.PublisherClient.SERVICE_ADDRESS,
+            port=publisher_client.PublisherClient.DEFAULT_SERVICE_PORT,
+        )
+
     def batch(self, topic, message, create=True, autocommit=True):
         """Return the current batch for the provided topic.
 
diff --git a/pubsub/google/cloud/pubsub_v1/subscriber/client.py b/pubsub/google/cloud/pubsub_v1/subscriber/client.py
--- a/pubsub/google/cloud/pubsub_v1/subscriber/client.py
+++ b/pubsub/google/cloud/pubsub_v1/subscriber/client.py
@@ -15,7 +15,11 @@
 from __future__ import absolute_import
 
 import pkg_resources
+import os
 
+import grpc
+
+from google.api_core import grpc_helpers
 from google.cloud.gapic.pubsub.v1 import subscriber_client
 
 from google.cloud.pubsub_v1 import _gapic
@@ -49,6 +53,29 @@ class in order to define your own consumer. This is primarily
             arguments.
     """
     def __init__(self, policy_class=thread.Policy, **kwargs):
+        # Sanity check: Is our goal to use the emulator?
+        # If so, create a grpc insecure channel with the emulator host
+        # as the target.
+        if os.environ.get('PUBSUB_EMULATOR_HOST'):
+            kwargs['channel'] = grpc.insecure_channel(
+                target=os.environ.get('PUBSUB_EMULATOR_HOST'),
+            )
+
+        # Use a custom channel.
+        # We need this in order to set appropriate default message size and
+        # keepalive options.
+        if 'channel' not in kwargs:
+            kwargs['channel'] = grpc_helpers.create_channel(
+                credentials=kwargs.get('credentials', None),
+                target=self.target,
+                scopes=subscriber_client.SubscriberClient._ALL_SCOPES,
+                options={
+                    'grpc.max_send_message_length': -1,
+                    'grpc.max_receive_message_length': -1,
+                    'grpc.keepalive_time_ms': 30000,
+                }.items(),
+            )
+
         # Add the metrics headers, and instantiate the underlying GAPIC
         # client.
         kwargs['lib_name'] = 'gccl'
@@ -59,6 +86,18 @@ def __init__(self, policy_class=thread.Policy, **kwargs):
         # messages.
         self._policy_class = policy_class
 
+    @property
+    def target(self):
+        """Return the target (where the API is).
+
+        Returns:
+            str: The location of the API.
+        """
+        return '{host}:{port}'.format(
+            host=subscriber_client.SubscriberClient.SERVICE_ADDRESS,
+            port=subscriber_client.SubscriberClient.DEFAULT_SERVICE_PORT,
+        )
+
     def subscribe(self, subscription, callback=None, flow_control=()):
         """Return a representation of an individual subscription.
 


Test patch:
diff --git a/pubsub/tests/unit/pubsub_v1/publisher/test_publisher_client.py b/pubsub/tests/unit/pubsub_v1/publisher/test_publisher_client.py
--- a/pubsub/tests/unit/pubsub_v1/publisher/test_publisher_client.py
+++ b/pubsub/tests/unit/pubsub_v1/publisher/test_publisher_client.py
@@ -12,6 +12,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from __future__ import absolute_import
+import os
+
 import mock
 
 import pytest
@@ -38,6 +41,18 @@ def test_init():
     assert client.batch_settings.max_messages == 1000
 
 
+def test_init_emulator(monkeypatch):
+    monkeypatch.setenv('PUBSUB_EMULATOR_HOST', '/foo/bar/')
+    client = create_client()
+
+    # Establish that a gRPC request would attempt to hit the emulator host.
+    #
+    # Sadly, there seems to be no good way to do this without poking at
+    # the private API of gRPC.
+    channel = client.api.publisher_stub.Publish._channel
+    assert channel.target().decode('utf8') == '/foo/bar/'
+
+
 def test_batch_accepting():
     """Establish that an existing batch is returned if it accepts messages."""
     client = create_client()
diff --git a/pubsub/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py b/pubsub/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py
--- a/pubsub/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py
+++ b/pubsub/tests/unit/pubsub_v1/subscriber/test_subscriber_client.py
@@ -29,6 +29,18 @@ def test_init():
     assert client._policy_class is thread.Policy
 
 
+def test_init_emulator(monkeypatch):
+    monkeypatch.setenv('PUBSUB_EMULATOR_HOST', '/baz/bacon/')
+    client = create_client()
+
+    # Establish that a gRPC request would attempt to hit the emulator host.
+    #
+    # Sadly, there seems to be no good way to do this without poking at
+    # the private API of gRPC.
+    channel = client.api.subscriber_stub.Pull._channel
+    assert channel.target().decode('utf8') == '/baz/bacon/'
+
+
 def test_subscribe():
     client = create_client()
     subscription = client.subscribe('sub_name_a')