internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ray-project/ray

Issue:
[docs/tune] Provide better documentation for reporter/results
### Describe the problem
The `reporter` object in function-based API is not well documented on the docs.

### Source code / logs
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/ray-dev/y4UXbSPoYaw/cLmdgq2nAwAJ



Discussion:

Resolving patch:
diff --git a/python/ray/tune/function_runner.py b/python/ray/tune/function_runner.py
--- a/python/ray/tune/function_runner.py
+++ b/python/ray/tune/function_runner.py
@@ -14,11 +14,12 @@
 
 
 class StatusReporter(object):
-    """Object passed into your main() that you can report status through.
+    """Object passed into your function that you can report status through.
 
     Example:
-        >>> reporter = StatusReporter()
-        >>> reporter(timesteps_total=1)
+        >>> def trainable_function(config, reporter):
+        >>>     assert isinstance(reporter, StatusReporter)
+        >>>     reporter(timesteps_total=1)
     """
 
     def __init__(self):
@@ -33,6 +34,9 @@ def __call__(self, **kwargs):
 
         Args:
             kwargs: Latest training result status.
+
+        Example:
+            >>> reporter(mean_accuracy=1, training_iteration=4)
         """
 
         with self._lock:
diff --git a/python/ray/tune/result.py b/python/ray/tune/result.py
--- a/python/ray/tune/result.py
+++ b/python/ray/tune/result.py
@@ -4,6 +4,8 @@
 
 import os
 
+# yapf: disable
+# __sphinx_doc_begin__
 # (Optional/Auto-filled) training is terminated. Filled only if not provided.
 DONE = "done"
 
@@ -37,6 +39,8 @@
 
 # (Auto-filled) The index of this training iteration.
 TRAINING_ITERATION = "training_iteration"
+# __sphinx_doc_end__
+# yapf: enable
 
 # Where Tune writes result files by default
 DEFAULT_RESULTS_DIR = (os.environ.get("TUNE_RESULT_DIR")


Test patch:
Repository: ray-project/ray

Issue:
[tune] Tune logger should indent format params.json
It would be nice if the tune params.json was automatically formatted. I keep going back to the variants fairly often and always have to manually (automatically) reformat it.


Discussion:

Resolving patch:
diff --git a/python/ray/tune/logger.py b/python/ray/tune/logger.py
--- a/python/ray/tune/logger.py
+++ b/python/ray/tune/logger.py
@@ -97,7 +97,12 @@ class _JsonLogger(Logger):
     def _init(self):
         config_out = os.path.join(self.logdir, "params.json")
         with open(config_out, "w") as f:
-            json.dump(self.config, f, sort_keys=True, cls=_SafeFallbackEncoder)
+            json.dump(
+                self.config,
+                f,
+                indent=2,
+                sort_keys=True,
+                cls=_SafeFallbackEncoder)
         local_file = os.path.join(self.logdir, "result.json")
         self.local_out = open(local_file, "w")
 


Test patch:
Repository: ray-project/ray

Issue:
Remove duplicate feed dict constructing
in `python/ray/experimental/sgd/sgd_worker.py` 

```python
    def compute_gradients(self):
        start = time.time()
        feed_dict = self._grad_feed_dict()
        # Aggregate feed dicts for each model on this worker.
        for model in self.models:
            feed_dict.update(model.get_feed_dict())
```

with `_grad_feed_dict` definitions:

```python
    def _grad_feed_dict(self):
        # Aggregate feed dicts for each model on this worker.
        feed_dict = {}
        for model in self.models:
            feed_dict.update(model.get_feed_dict())
        return feed_dict
```


Discussion:

Resolving patch:
diff --git a/python/ray/experimental/sgd/sgd_worker.py b/python/ray/experimental/sgd/sgd_worker.py
--- a/python/ray/experimental/sgd/sgd_worker.py
+++ b/python/ray/experimental/sgd/sgd_worker.py
@@ -205,9 +205,6 @@ def for_model(self, fn):
     def compute_gradients(self):
         start = time.time()
         feed_dict = self._grad_feed_dict()
-        # Aggregate feed dicts for each model on this worker.
-        for model in self.models:
-            feed_dict.update(model.get_feed_dict())
         # We only need to fetch the first per_device_grad, since they are
         # averaged across all devices by allreduce.
         fetches = self.sess.run(


Test patch:
Repository: ray-project/ray

Issue:
Fix formatting of PyPI package description.
See https://pypi.org/project/ray/.

Note that we can test this out first at https://test.pypi.org/project/ray/.


Discussion:

Resolving patch:
diff --git a/python/setup.py b/python/setup.py
--- a/python/setup.py
+++ b/python/setup.py
@@ -154,6 +154,8 @@ def find_version(*filepath):
 setup(
     name="ray",
     version=find_version("ray", "__init__.py"),
+    author="Ray Team",
+    author_email="ray-dev@googlegroups.com",
     description=("A system for parallel and distributed Python that unifies "
                  "the ML ecosystem."),
     long_description=open("../README.rst").read(),


Test patch:
Repository: ray-project/ray

Issue:
[modin] Importing Modin before Ray can sometimes cause ImportError

### Describe the problem
<!-- Describe the problem clearly here. -->
When running Modin with Ray installed from source, I am sometimes running into `ImportError` and `ModuleNotFoundError` which is occurring when I am running a modified version of Modin. This forces me to modify Ray's source such that it does not try to use the Modin that is bundled with Ray.

I will work on a solution for this.

### Source code / logs

`import modin.pandas as pd`
```
Traceback (most recent call last):
  File "/home/ubuntu/ray/python/ray/function_manager.py", line 165, in fetch_and_register_remote_function
    function = pickle.loads(serialized_function)
ModuleNotFoundError: No module named 'modin.data_management.utils'
```


Discussion:
Is that exception raised on a worker or the driver?
It is raised in every worker immediately after import.

Resolving patch:
diff --git a/python/ray/__init__.py b/python/ray/__init__.py
--- a/python/ray/__init__.py
+++ b/python/ray/__init__.py
@@ -47,7 +47,7 @@
     raise
 
 modin_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), "modin")
-sys.path.insert(0, modin_path)
+sys.path.append(modin_path)
 
 from ray.raylet import ObjectID, _config  # noqa: E402
 from ray.profiling import profile  # noqa: E402


Test patch:
Repository: ray-project/ray

Issue:
Actor definition needs Actor(object).
The following code does not work on Python 2 but works on Python 3. 

```
import ray
ray.init()

@ray.remote
class Outer():
    def __init__(self):
        pass
```
Error thrown:
```
Traceback (most recent call last):
  File "/Users/rliaw/miniconda2/envs/py2/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/Users/rliaw/miniconda2/envs/py2/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/rliaw/miniconda2/envs/py2/lib/python2.7/site-packages/ray/worker.py", line 1166, in import_thread
    worker.fetch_and_register_actor(key, worker)
  File "/Users/rliaw/miniconda2/envs/py2/lib/python2.7/site-packages/ray/actor.py", line 86, in fetch_and_register_actor
    worker.actors[actor_id_str] = unpickled_class.__new__(unpickled_class)
AttributeError: class Class has no attribute '__new__'
```



Discussion:
Do we want to support old-style classes?

We should certainly give a better error message.
Better error message: Yes! It's ok to not support old-style classes for actors I think. I tagged it with "help wanted" as it looks like a low hanging fruit that would be perfect for somebody to look into to contribute to the project.

Resolving patch:
diff --git a/python/ray/actor.py b/python/ray/actor.py
--- a/python/ray/actor.py
+++ b/python/ray/actor.py
@@ -779,6 +779,13 @@ def __setstate__(self, state):
 
 def make_actor(cls, num_cpus, num_gpus, resources, actor_method_cpus,
                checkpoint_interval, max_reconstructions):
+    # Give an error if cls is an old-style class.
+    if not issubclass(cls, object):
+        raise TypeError(
+            "The @ray.remote decorator cannot be applied to old-style "
+            "classes. In Python 2, you must declare the class with "
+            "'class ClassName(object):' instead of 'class ClassName:'.")
+
     if checkpoint_interval is None:
         checkpoint_interval = -1
     if max_reconstructions is None:


Test patch:
diff --git a/test/actor_test.py b/test/actor_test.py
--- a/test/actor_test.py
+++ b/test/actor_test.py
@@ -98,6 +98,16 @@ def foo(self):
         ray.get(actor.foo.remote())
 
 
+@pytest.mark.skipif(
+    sys.version_info >= (3, 0), reason="This test requires Python 2.")
+def test_old_style_error(ray_start_regular):
+    with pytest.raises(TypeError):
+
+        @ray.remote
+        class Actor:
+            pass
+
+
 def test_keyword_args(ray_start_regular):
     @ray.remote
     class Actor(object):
Repository: ray-project/ray

Issue:
Too many warning messages when actor handles are passed into tasks.
```python
import ray

ray.init()

@ray.remote
class Foo:
    pass

@ray.remote
def g(handle):
    pass

f = Foo.remote()

g.remote(f)
```

The last line will print out

```
WARNING: Falling back to serializing objects of type <class 'ray._raylet.ActorID'> by using pickle. This may be inefficient.
WARNING: Falling back to serializing objects of type <class 'ray._raylet.ActorHandleID'> by using pickle. This may be inefficient.
WARNING: Falling back to serializing objects of type <class 'ray._raylet.DriverID'> by using pickle. This may be inefficient.
```

This was presumably introduced in #3541. We should just register pickle as a custom serializer for all ID types ahead of time.

cc @suquark 


Discussion:
Yes, it should be. Let me create a PR to fix it.

Resolving patch:
diff --git a/python/ray/__init__.py b/python/ray/__init__.py
--- a/python/ray/__init__.py
+++ b/python/ray/__init__.py
@@ -51,7 +51,7 @@
 
 from ray._raylet import (UniqueID, ObjectID, DriverID, ClientID, ActorID,
                          ActorHandleID, FunctionID, ActorClassID, TaskID,
-                         Config as _Config)  # noqa: E402
+                         _ID_TYPES, Config as _Config)  # noqa: E402
 
 _config = _Config()
 
@@ -77,7 +77,8 @@
     "remote", "profile", "actor", "method", "get_gpu_ids", "get_resource_ids",
     "get_webui_url", "register_custom_serializer", "shutdown",
     "is_initialized", "SCRIPT_MODE", "WORKER_MODE", "LOCAL_MODE",
-    "PYTHON_MODE", "global_state", "_config", "__version__", "internal"
+    "PYTHON_MODE", "global_state", "_config", "__version__", "internal",
+    "_ID_TYPES"
 ]
 
 __all__ += [
diff --git a/python/ray/worker.py b/python/ray/worker.py
--- a/python/ray/worker.py
+++ b/python/ray/worker.py
@@ -1099,22 +1099,11 @@ def _initialize_serialization(driver_id, worker=global_worker):
     serialization_context.set_pickle(pickle.dumps, pickle.loads)
     pyarrow.register_torch_serialization_handlers(serialization_context)
 
-    # Define a custom serializer and deserializer for handling Object IDs.
-    def object_id_custom_serializer(obj):
-        return obj.binary()
-
-    def object_id_custom_deserializer(serialized_obj):
-        return ObjectID(serialized_obj)
-
-    # We register this serializer on each worker instead of calling
-    # register_custom_serializer from the driver so that isinstance still
-    # works.
-    serialization_context.register_type(
-        ObjectID,
-        "ray.ObjectID",
-        pickle=False,
-        custom_serializer=object_id_custom_serializer,
-        custom_deserializer=object_id_custom_deserializer)
+    for id_type in ray._ID_TYPES:
+        serialization_context.register_type(
+            id_type,
+            "{}.{}".format(id_type.__module__, id_type.__name__),
+            pickle=True)
 
     def actor_handle_serializer(obj):
         return obj._serialization_helper(True)


Test patch:
Repository: ray-project/ray

Issue:
[rllib] Question regarding twin-Q usage in DDPG
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:  Linux Ubuntu 16.04
- **Ray installed from (source or binary)**: source
- **Ray version**: latest
- **Python version**: 3.6
- **Exact command to reproduce**:

### Describe the problem
When activating the `twin_q` functionality in DDPG the following loss specific operations are defined when creating the policy graph (see `ActorCriticLoss`):

```
q_t_selected = tf.squeeze(q_t, axis=len(q_t.shape) - 1)
if twin_q:  
    twin_q_t_selected = tf.squeeze(q_t, axis=len(q_t.shape) - 1)  
    q_tp1 = tf.minimum(q_tp1, twin_q_tp1)  
```
In this case `q_t_selected` and `twin_q_t_selected` hold the same operation. Probably `twin_q_t_selected` need to be adjusted the following way:
```
twin_q_t_selected = tf.squeeze(twin_q_t, axis=len(q_t.shape) - 1)  
```





Discussion:
Yes, that looks like a bug. cc @joneswong 
yes, we must optimize both of the two q-net. twin_q_t should be used here.

Resolving patch:
diff --git a/python/ray/rllib/agents/ddpg/ddpg_policy_graph.py b/python/ray/rllib/agents/ddpg/ddpg_policy_graph.py
--- a/python/ray/rllib/agents/ddpg/ddpg_policy_graph.py
+++ b/python/ray/rllib/agents/ddpg/ddpg_policy_graph.py
@@ -138,7 +138,7 @@ def __init__(self,
 
         q_t_selected = tf.squeeze(q_t, axis=len(q_t.shape) - 1)
         if twin_q:
-            twin_q_t_selected = tf.squeeze(q_t, axis=len(q_t.shape) - 1)
+            twin_q_t_selected = tf.squeeze(twin_q_t, axis=len(q_t.shape) - 1)
             q_tp1 = tf.minimum(q_tp1, twin_q_tp1)
 
         q_tp1_best = tf.squeeze(input=q_tp1, axis=len(q_tp1.shape) - 1)


Test patch:
Repository: ray-project/ray

Issue:
dashboard.py is not packaged in the Linux Ray wheels.
See the conversation in https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/ray-dev/M8wGAdEhkTw/QIbvbuoJBAAJ.

I think we can fix this just by putting `__init__.py` in the `ray/python/dashboard` directory, though we have to make sure that that includes the html and javascript files.

cc @virtualluke


Discussion:

Resolving patch:
diff --git a/python/setup.py b/python/setup.py
--- a/python/setup.py
+++ b/python/setup.py
@@ -24,7 +24,9 @@
     "ray/core/src/ray/gcs/redis_module/libray_redis_module.so",
     "ray/core/src/plasma/plasma_store_server", "ray/_raylet.so",
     "ray/core/src/ray/raylet/raylet_monitor", "ray/core/src/ray/raylet/raylet",
-    "ray/WebUI.ipynb"
+    "ray/WebUI.ipynb", "ray/dashboard/dashboard.py",
+    "ray/dashboard/index.html", "ray/dashboard/res/main.css",
+    "ray/dashboard/res/main.js"
 ]
 
 # These are the directories where automatically generated Python flatbuffer


Test patch:
Repository: ray-project/ray

Issue:
[tune] Open file handles but never close them in Trainable
This file is cluttered with functions opening file handles but never closing them

https://github.com/ray-project/ray/blob/master/python/ray/tune/trainable.py


Discussion:
Aren't they closed automatically by python ref counting?
They are garbage collected eventually, but it is bad practice and results in a bunch of warnings in my Std error.
https://stackoverflow.com/q/1832528/1116010
I see, that makes sense to clean up then.

Resolving patch:
diff --git a/python/ray/tune/trainable.py b/python/ray/tune/trainable.py
--- a/python/ray/tune/trainable.py
+++ b/python/ray/tune/trainable.py
@@ -245,14 +245,15 @@ def save(self, checkpoint_dir=None):
             raise ValueError(
                 "`_save` must return a dict or string type: {}".format(
                     str(type(checkpoint))))
-        pickle.dump({
-            "experiment_id": self._experiment_id,
-            "iteration": self._iteration,
-            "timesteps_total": self._timesteps_total,
-            "time_total": self._time_total,
-            "episodes_total": self._episodes_total,
-            "saved_as_dict": saved_as_dict
-        }, open(checkpoint_path + ".tune_metadata", "wb"))
+        with open(checkpoint_path + ".tune_metadata", "wb") as f:
+            pickle.dump({
+                "experiment_id": self._experiment_id,
+                "iteration": self._iteration,
+                "timesteps_total": self._timesteps_total,
+                "time_total": self._time_total,
+                "episodes_total": self._episodes_total,
+                "saved_as_dict": saved_as_dict
+            }, f)
         return checkpoint_path
 
     def save_to_object(self):
@@ -271,7 +272,8 @@ def save_to_object(self):
         for path in os.listdir(base_dir):
             path = os.path.join(base_dir, path)
             if path.startswith(checkpoint_prefix):
-                data[os.path.basename(path)] = open(path, "rb").read()
+                with open(path, "rb") as f:
+                    data[os.path.basename(path)] = f.read()
 
         out = io.BytesIO()
         data_dict = pickle.dumps({
@@ -294,7 +296,8 @@ def restore(self, checkpoint_path):
         This method restores additional metadata saved with the checkpoint.
         """
 
-        metadata = pickle.load(open(checkpoint_path + ".tune_metadata", "rb"))
+        with open(checkpoint_path + ".tune_metadata", "rb") as f:
+            metadata = pickle.load(f)
         self._experiment_id = metadata["experiment_id"]
         self._iteration = metadata["iteration"]
         self._timesteps_total = metadata["timesteps_total"]


Test patch:
Repository: ray-project/ray

Issue:
[tune] Add `--output` to the Tune docs
We should add --output to the docs.

_Originally posted by @richardliaw in https://github.com/ray-project/ray/pull/4322#issuecomment-477903993_

cc @andrewztan


Discussion:

Resolving patch:
diff --git a/python/ray/tune/commands.py b/python/ray/tune/commands.py
--- a/python/ray/tune/commands.py
+++ b/python/ray/tune/commands.py
@@ -194,16 +194,14 @@ def list_trials(experiment_path,
     print_format_output(checkpoints_df)
 
     if output:
-        experiment_path = os.path.expanduser(experiment_path)
-        output_path = os.path.join(experiment_path, output)
         file_extension = os.path.splitext(output)[1].lower()
         if file_extension in (".p", ".pkl", ".pickle"):
-            checkpoints_df.to_pickle(output_path)
+            checkpoints_df.to_pickle(output)
         elif file_extension == ".csv":
-            checkpoints_df.to_csv(output_path, index=False)
+            checkpoints_df.to_csv(output, index=False)
         else:
             raise ValueError("Unsupported filetype: {}".format(output))
-        print("Output saved at:", output_path)
+        print("Output saved at:", output)
 
 
 def list_experiments(project_path,
@@ -295,15 +293,14 @@ def list_experiments(project_path,
     print_format_output(info_df)
 
     if output:
-        output_path = os.path.join(base, output)
         file_extension = os.path.splitext(output)[1].lower()
         if file_extension in (".p", ".pkl", ".pickle"):
-            info_df.to_pickle(output_path)
+            info_df.to_pickle(output)
         elif file_extension == ".csv":
-            info_df.to_csv(output_path, index=False)
+            info_df.to_csv(output, index=False)
         else:
             raise ValueError("Unsupported filetype: {}".format(output))
-        print("Output saved at:", output_path)
+        print("Output saved at:", output)
 
 
 def add_note(path, filename="note.txt"):


Test patch:
Repository: ray-project/ray

Issue:
Task arguments are retrieved serially from the object store.
@stephanie-wang noticed that when we do `ray.get`, we fetch objects in parallel from the object store:

https://github.com/ray-project/ray/blob/e05baed336470c6568c0f32017ed1fdde9a17132/python/ray/worker.py#L2510-L2511

But when we get the arguments for a task, we get them serially:

https://github.com/ray-project/ray/blob/e05baed336470c6568c0f32017ed1fdde9a17132/python/ray/worker.py#L850-L853

For settings when many object IDs are passed into a remote function, this can slow things down.


Discussion:

Resolving patch:
diff --git a/python/ray/worker.py b/python/ray/worker.py
--- a/python/ray/worker.py
+++ b/python/ray/worker.py
@@ -782,18 +782,27 @@ def _get_arguments_for_execution(self, function_name, serialized_args):
             RayError: This exception is raised if a task that
                 created one of the arguments failed.
         """
-        arguments = []
+        arguments = [None] * len(serialized_args)
+        object_ids = []
+        object_indices = []
+
         for (i, arg) in enumerate(serialized_args):
             if isinstance(arg, ObjectID):
-                # get the object from the local object store
-                argument = self.get_object([arg])[0]
-                if isinstance(argument, RayError):
-                    raise argument
+                object_ids.append(arg)
+                object_indices.append(i)
             else:
                 # pass the argument by value
-                argument = arg
+                arguments[i] = arg
+
+        # Get the objects from the local object store.
+        if len(object_ids) > 0:
+            values = self.get_object(object_ids)
+            for i, value in enumerate(values):
+                if isinstance(value, RayError):
+                    raise value
+                else:
+                    arguments[object_indices[i]] = value
 
-            arguments.append(argument)
         return arguments
 
     def _store_outputs_in_object_store(self, object_ids, outputs):


Test patch:
Repository: ray-project/ray

Issue:
Bumping to 0.8.0.dev1?
@robertnishihara @devin-petersohn Is there a followup PR to bump back to 0.8.0.dev1?

_Originally posted by @richardliaw in https://github.com/ray-project/ray/pull/4890#issuecomment-498979267_


Discussion:

Resolving patch:
diff --git a/python/ray/__init__.py b/python/ray/__init__.py
--- a/python/ray/__init__.py
+++ b/python/ray/__init__.py
@@ -96,7 +96,7 @@
 from ray.runtime_context import _get_runtime_context  # noqa: E402
 
 # Ray version string.
-__version__ = "0.7.1"
+__version__ = "0.8.0.dev1"
 
 __all__ = [
     "global_state",


Test patch: