Repository: googleapis/google-cloud-python

Issue:
Pubsub unit test hangs intermittently
See: https://circleci.com/gh/GoogleCloudPlatform/google-cloud-python/6432

```
if [[ -n $(grep pubsub ~/target_packages) ]]; then
  nox -f pubsub/nox.py
fi

nox > Running session default
nox > virtualenv /var/code/gcp/.nox/default
nox > chdir /var/code/gcp/pubsub
nox > pip install --upgrade mock pytest pytest-cov ../api_core ../core
nox > pip install --upgrade -e .
nox > py.test --quiet --cov-append --cov-report= --cov=google.cloud.pubsub --cov=google.cloud.pubsub_v1 --cov-config=.coveragerc tests/unit
........................................................................ [ 28%]
........................................................................ [ 57%]
........................................................................ [ 86%]
..................................                                       [100%]


250 passed in 4.14 seconds
nox > Session default successful. :)
nox > Running session unit(py='2.7')
nox > virtualenv /var/code/gcp/.nox/unit-2-7 -p python2.7
nox > chdir /var/code/gcp/pubsub
nox > pip install --upgrade mock pytest pytest-cov ../api_core ../core
nox > pip install --upgrade -e .
nox > py.test --quiet --cov-append --cov-report= --cov=google.cloud.pubsub --cov=google.cloud.pubsub_v1 --cov-config=.coveragerc tests/unit
........................................................................ [ 28%]
...............................................................Too long with no output (exceeded 10m0s)
```


Discussion:
That's the unit test, is it not?
@jonparrott Yes.  I updated the issue title when I figured that out (which is when I bumped the priority, too).
Oops, looks like I PEBKAC'ed the save of that change until just now. :(
I've been running the tests over and over for like half an hour now and can't reproduce this yet.

Resolving patch:
diff --git a/pubsub/google/cloud/pubsub_v1/subscriber/_protocol/bidi.py b/pubsub/google/cloud/pubsub_v1/subscriber/_protocol/bidi.py
--- a/pubsub/google/cloud/pubsub_v1/subscriber/_protocol/bidi.py
+++ b/pubsub/google/cloud/pubsub_v1/subscriber/_protocol/bidi.py
@@ -421,16 +421,24 @@ def _thread_main(self):
             self._bidi_rpc.open()
 
             while self._bidi_rpc.is_active:
-                if not self.is_paused:
-                    _LOGGER.debug('waiting for recv.')
-                    response = self._bidi_rpc.recv()
-                    _LOGGER.debug('recved response.')
-                    self._on_response(response)
-                else:
-                    _LOGGER.debug('paused, waiting for waking.')
-                    with self._wake:
+                # Do not allow the paused status to change at all during this
+                # section. There is a condition where we could be resumed
+                # between checking if we are paused and calling wake.wait(),
+                # which means that we will miss the notification to wake up
+                # (oops!) and wait for a notification that will never come.
+                # Keeping the lock throughout avoids that.
+                # In the future, we could use `Condition.wait_for` if we drop
+                # Python 2.7.
+                with self._wake:
+                    if self._paused:
+                        _LOGGER.debug('paused, waiting for waking.')
                         self._wake.wait()
-                    _LOGGER.debug('woken.')
+                        _LOGGER.debug('woken.')
+
+                _LOGGER.debug('waiting for recv.')
+                response = self._bidi_rpc.recv()
+                _LOGGER.debug('recved response.')
+                self._on_response(response)
 
         except exceptions.GoogleAPICallError as exc:
             _LOGGER.debug(


Test patch:
diff --git a/pubsub/tests/unit/pubsub_v1/subscriber/test_bidi.py b/pubsub/tests/unit/pubsub_v1/subscriber/test_bidi.py
--- a/pubsub/tests/unit/pubsub_v1/subscriber/test_bidi.py
+++ b/pubsub/tests/unit/pubsub_v1/subscriber/test_bidi.py
@@ -504,7 +504,9 @@ def on_response(response):
         # consumer.
         should_continue.set()
         consumer.resume()
+
         responses_and_events[mock.sentinel.response_2].wait()
+
         assert recved_responses == [
             mock.sentinel.response_1, mock.sentinel.response_2]