Repository: ray-project/ray

Issue:
Non-integer timeout for signal.receive causes malformed redis query
<!--
General questions should be asked on the mailing list ray-dev@googlegroups.com.
Questions about how to use Ray should be asked on
[StackOverflow](https://stackoverflow.com/questions/tagged/ray).

Before submitting an issue, please fill out the following form.
-->

### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Ubuntu 18.04
- **Ray installed from (source or binary)**: source
- **Ray version**: 0.7.0.dev2
- **Python version**: 3.7.2

### Describe the problem
The timeout argument in `signal.receive()` does not work with non-integer values. For instance, calling `signal.receive(sources, timeout=0.01)` causes a `redis.exceptions.ResponseError: timeout is not an integer or out of range`.

This is because of L131 in `signal.py`, where the timeout is converted to ms and then converted to a string - redis expects an int, but if you pass a double to the method (like 0.1), the result of `str(1000 * timeout)` is  `100.0`. The correct string would have been `100`. A fix would be to change `str(1000 * timeout)` to `str(int(1000 * timeout))` and have checks to ensure timeout is not < 1000.

https://github.com/ray-project/ray/blob/d951eb740ffe22c385d75df62aa18da790706804/python/ray/experimental/signal.py#L129-L133

### Source code / logs
Traceback: 
```
Traceback (most recent call last):
  File "driver.py", line 21, in <module>
    signals = signal.receive(self.clients, timeout=0.01)
  File "/home/romilb/ray/python/ray/experimental/signal.py", line 141, in receive
    answers = ray.worker.global_worker.redis_client.execute_command(query)
  File "/home/romilb/anaconda3/lib/python3.7/site-packages/redis/client.py", line 668, in execute_command
    return self.parse_response(connection, command_name, **options)
  File "/home/romilb/anaconda3/lib/python3.7/site-packages/redis/client.py", line 680, in parse_response
    response = connection.read_response()
  File "/home/romilb/anaconda3/lib/python3.7/site-packages/redis/connection.py", line 629, in read_response
    raise response
redis.exceptions.ResponseError: timeout is not an integer or out of range
```
I also printed the corresponding redis query:

```
(pid=31330) XREAD BLOCK 10.0 STREAMS fc96f993802bb5a31b5868e69179fe76b62b7c3b 0
```



Discussion:
I'm experiencing this problem. Seems like an easy fix, is this all that needs to happen?
Yes, do you want to submit a PR?
I do

On Wed, Jun 19, 2019, 4:06 PM Philipp Moritz <notifications@github.com>
wrote:

> Yes, do you want to submit a PR?
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub
> <https://github.com/ray-project/ray/issues/4674?email_source=notifications&email_token=ACDIN2RSR35IO4D3K6MZI7DP3KGVLA5CNFSM4HHJEO32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYDEFXA#issuecomment-503726812>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/ACDIN2RVVWSIZC622HFFQTLP3KGVLANCNFSM4HHJEO3Q>
> .
>


Resolving patch:
diff --git a/python/ray/experimental/signal.py b/python/ray/experimental/signal.py
--- a/python/ray/experimental/signal.py
+++ b/python/ray/experimental/signal.py
@@ -2,6 +2,8 @@
 from __future__ import division
 from __future__ import print_function
 
+import logging
+
 from collections import defaultdict
 
 import ray
@@ -13,6 +15,8 @@
 # in node_manager.cc
 ACTOR_DIED_STR = "ACTOR_DIED_SIGNAL"
 
+logger = logging.getLogger(__name__)
+
 
 class Signal(object):
     """Base class for Ray signals."""
@@ -125,10 +129,16 @@ def receive(sources, timeout=None):
     for s in sources:
         task_id_to_sources[_get_task_id(s).hex()].append(s)
 
+    if timeout < 1e-3:
+        logger.warning("Timeout too small. Using 1ms minimum")
+        timeout = 1e-3
+
+    timeout_ms = int(1000 * timeout)
+
     # Construct the redis query.
     query = "XREAD BLOCK "
-    # Multiply by 1000x since timeout is in sec and redis expects ms.
-    query += str(1000 * timeout)
+    # redis expects ms.
+    query += str(timeout_ms)
     query += " STREAMS "
     query += " ".join([task_id for task_id in task_id_to_sources])
     query += " "


Test patch:
diff --git a/python/ray/tests/test_signal.py b/python/ray/tests/test_signal.py
--- a/python/ray/tests/test_signal.py
+++ b/python/ray/tests/test_signal.py
@@ -353,3 +353,36 @@ def f(sources):
     assert len(result_list) == 1
     result_list = ray.get(f.remote([a]))
     assert len(result_list) == 1
+
+
+def test_non_integral_receive_timeout(ray_start_regular):
+    @ray.remote
+    def send_signal(value):
+        signal.send(UserSignal(value))
+
+    a = send_signal.remote(0)
+    # make sure send_signal had a chance to execute
+    ray.get(a)
+
+    result_list = ray.experimental.signal.receive([a], timeout=0.1)
+
+    assert len(result_list) == 1
+
+
+def test_small_receive_timeout(ray_start_regular):
+    """ Test that receive handles timeout smaller than the 1ms min
+    """
+    # 0.1 ms
+    small_timeout = 1e-4
+
+    @ray.remote
+    def send_signal(value):
+        signal.send(UserSignal(value))
+
+    a = send_signal.remote(0)
+    # make sure send_signal had a chance to execute
+    ray.get(a)
+
+    result_list = ray.experimental.signal.receive([a], timeout=small_timeout)
+
+    assert len(result_list) == 1