internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: ray-project/ray

Issue:
[tune] Parameters from `tune.choice()` do not get logged to TensorBoard when integers
### What is the problem?
When providing parameters via `tune.choice()` that include integers, the values are not logged to TensorBoard's HPARAMS section. 
The issue is that `numpy.random.choice([1, 2, 3])` (for example) returns `numpy.int32`/`numpy.int64` and those types are not included in the `VALID_HPARAMS = (str, bool, int, float, list)` tuple (python/ray/tune/logger.py). 
Since TensorBoard has no issues with logging `numpy.int32/64`, one simple solution would be to just include those types in the tuple above. Happy to provide a PR if you think this is the way to go.

*Ray version and other system information (Python version, TensorFlow version, OS):*
ray: 0.8.6
python: 3.7.7
tensorboard: 2.2.2
ubuntu: 20.04

### Reproduction (REQUIRED)
```python
from ray import tune

def trainable(config):
    tune.report(score=config["a"])

config_dict = {"a": tune.choice([1, 2, 3])}

tune.run(trainable, config=config_dict, num_samples=1)
```
- [x] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:
yes! that'd be great - could you push a PR and ping me?

Resolving patch:
diff --git a/python/ray/tune/logger.py b/python/ray/tune/logger.py
--- a/python/ray/tune/logger.py
+++ b/python/ray/tune/logger.py
@@ -187,7 +187,7 @@ class TBXLogger(Logger):
     """
 
     # NoneType is not supported on the last TBX release yet.
-    VALID_HPARAMS = (str, bool, int, float, list)
+    VALID_HPARAMS = (str, bool, np.bool8, int, np.integer, float, list)
 
     def _init(self):
         try:


Test patch:
diff --git a/python/ray/tune/tests/test_logger.py b/python/ray/tune/tests/test_logger.py
--- a/python/ray/tune/tests/test_logger.py
+++ b/python/ray/tune/tests/test_logger.py
@@ -2,6 +2,7 @@
 import unittest
 import tempfile
 import shutil
+import numpy as np
 
 from ray.tune.logger import JsonLogger, CSVLogger, TBXLogger
 
@@ -46,7 +47,17 @@ def testJSON(self):
         logger.close()
 
     def testTBX(self):
-        config = {"a": 2, "b": [1, 2], "c": {"c": {"D": 123}}}
+        config = {
+            "a": 2,
+            "b": [1, 2],
+            "c": {
+                "c": {
+                    "D": 123
+                }
+            },
+            "d": np.int64(1),
+            "e": np.bool8(True)
+        }
         t = Trial(evaluated_params=config, trial_id="tbx")
         logger = TBXLogger(config=config, logdir=self.test_dir, trial=t)
         logger.on_result(result(0, 4))
Repository: ray-project/ray

Issue:
[tune][rllib] Windows: GPU not recognized
### What is the problem?

I'm getting `ray.tune.error.TuneError: Insufficient cluster resources to launch trial`.
I specified a GPU in my config but ray does not recognize my GPU (RTX 2080) and throws an error.
I can get passed this by setting `num_gpus: 0` in my config for now.

https://gist.github.com/juliusfrost/fa7ebbb8d1dfc66eea0bbc4babcbe5aa

### Reproduction (REQUIRED)

```
git clone https://github.com/juliusfrost/rllib-tune-atari.git
cd rllib-tune-atari
pip install -r requirements.txt
python train.py --algo a2c
```

- [x] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:
#9114 @mehrdadn 
@mehrdadn Did you get a chance to look at this? I updated the script to reproduce to remove external dependencies.

Resolving patch:
diff --git a/python/ray/resource_spec.py b/python/ray/resource_spec.py
--- a/python/ray/resource_spec.py
+++ b/python/ray/resource_spec.py
@@ -3,6 +3,8 @@
 import logging
 import multiprocessing
 import os
+import subprocess
+import sys
 
 import ray
 import ray.ray_constants as ray_constants
@@ -229,12 +231,23 @@ def resolve(self, is_head, node_ip_address=None):
 def _autodetect_num_gpus():
     """Attempt to detect the number of GPUs on this machine.
 
-    TODO(rkn): This currently assumes Nvidia GPUs and Linux.
+    TODO(rkn): This currently assumes NVIDIA GPUs on Linux.
+    TODO(mehrdadn): This currently does not work on macOS.
+    TODO(mehrdadn): Use a better mechanism for Windows.
+
+    Possibly useful: tensorflow.config.list_physical_devices()
 
     Returns:
         The number of GPUs if any were detected, otherwise 0.
     """
-    proc_gpus_path = "/proc/driver/nvidia/gpus"
-    if os.path.isdir(proc_gpus_path):
-        return len(os.listdir(proc_gpus_path))
-    return 0
+    result = 0
+    if sys.platform.startswith("linux"):
+        proc_gpus_path = "/proc/driver/nvidia/gpus"
+        if os.path.isdir(proc_gpus_path):
+            result = len(os.listdir(proc_gpus_path))
+    elif sys.platform == "win32":
+        props = "AdapterCompatibility"
+        cmdargs = ["WMIC", "PATH", "Win32_VideoController", "GET", props]
+        lines = subprocess.check_output(cmdargs).splitlines()[1:]
+        result = len([l.rstrip() for l in lines if l.startswith(b"NVIDIA")])
+    return result


Test patch:
Repository: ray-project/ray

Issue:
[RLlib] Mutiagent learning: can't combine replay lockstep and multiple agents controlled by the same policy.
Hello,

While implementing MADDPG for PyTorch I noticed that it is not possible to combine the lockstep replay mode configuration and having multiple agents controlled by the same policy. This is due to the implementation of MultiAgentBatch as a dict of `PolicyID -> SampleBatch`. In the contrib TensorFlow implementation this issue was circumvented using parameter sharing. Another solution which is framework agnostic would be to group the agents. 

I think that this combination should be supported without needing to group the agents. However, I suspect that it would require significant code change. Maybe another solution exists?

Also, the documentation should be updated alongside with the code that checks that the config is valid.

Thanks 
(Amazing work btw)


Discussion:
Hmm, I think this should be ok. The batch will indeed be keyed by policy id, but you can look into the "agent_id" field of the batch to see which experiences correspond with which original agent.
Okk I will look into that next week and let you know. Thx

Resolving patch:
diff --git a/rllib/policy/sample_batch.py b/rllib/policy/sample_batch.py
--- a/rllib/policy/sample_batch.py
+++ b/rllib/policy/sample_batch.py
@@ -438,8 +438,8 @@ def timeslices(self, k: int) -> List["MultiAgentBatch"]:
         steps = []
         for policy_id, batch in self.policy_batches.items():
             for row in batch.rows():
-                steps.append((row[SampleBatch.EPS_ID], row["t"], policy_id,
-                              row))
+                steps.append((row[SampleBatch.EPS_ID], row["t"],
+                              row["agent_index"], policy_id, row))
         steps.sort()
 
         finished_slices = []
@@ -458,7 +458,7 @@ def finish_slice():
         # For each unique env timestep.
         for _, group in itertools.groupby(steps, lambda x: x[:2]):
             # Accumulate into the current slice.
-            for _, _, policy_id, row in group:
+            for _, _, _, policy_id, row in group:
                 cur_slice[policy_id].add_values(**row)
             cur_slice_size += 1
             # Slice has reached target number of env steps.


Test patch:
Repository: ray-project/ray

Issue:
[Tune] ProgressReporter default_columns is shared across instances
### What is the problem?
When adding a new column to a ProgressReporter that used the `DEFAULT_COLUMNS` during its creation, this column will be added to the actual default dictionary. This can cause several unexpected behaviors.
For instance, when creating several experiments with custom columns for each of them, using the same column multiple times will break (see reproduction).

### Reproduction (REQUIRED)
```python
from ray.tune import CLIReporter

reporter = CLIReporter()
reporter.add_metric_column('custom column')
reporter = CLIReporter()
reporter.add_metric_column('custom column')
```
Thows a `ValueError: Column custom column already exists.`

- [x] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).

### Solution
A solution is to copy this default parameter each time it is used [here](https://github.com/ray-project/ray/blob/4b31b383f379dad5514f4d93a9776db6b3caa38c/python/ray/tune/progress_reporter.py#L90).
I can send a PR is this is okay for you.


Discussion:
Sorry; I totally missed your comment at the bottom. A PR would be great!

Resolving patch:
diff --git a/python/ray/tune/progress_reporter.py b/python/ray/tune/progress_reporter.py
--- a/python/ray/tune/progress_reporter.py
+++ b/python/ray/tune/progress_reporter.py
@@ -87,7 +87,7 @@ def __init__(self,
                  max_progress_rows=20,
                  max_error_rows=20,
                  max_report_frequency=5):
-        self._metric_columns = metric_columns or self.DEFAULT_COLUMNS
+        self._metric_columns = metric_columns or self.DEFAULT_COLUMNS.copy()
         self._parameter_columns = parameter_columns or []
         self._max_progress_rows = max_progress_rows
         self._max_error_rows = max_error_rows


Test patch:
Repository: ray-project/ray

Issue:
[core] Raylets in local cluster do not log to separate files
<!--Please include [tune], [rllib], [autoscaler] etc. in the issue title if relevant-->

### What is the problem?

*Ray version and other system information (Python version, TensorFlow version, OS):* 0.9dev

When multiple raylets are started locally, it used to be that they would each log to a separate file, e.g., `raylet.1.out`, `raylet.2.out`, etc. Now, they all log to `raylet.out`, which makes it harder to debug.

### Reproduction (REQUIRED)

Any checked in pytest that starts a local cluster. For example, `RAY_BACKEND_LOG_LEVEL=debug pytest -sv python/ray/tests/test_advanced.py::test_wait_cluster`

- [x] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:
I believe this was caused by #8885, @wuisawesome.
@stephanie-wang That was another regression I noticed, adding a `unique=True` argument to `get_log_file_names` [here](https://github.com/ray-project/ray/blob/5881417ec4a993194ba6d64b156ad6b5ffbca97c/python/ray/node.py#L650) should do the trick:
```
raylet_out_name, raylet_err_name = self.get_log_file_names("raylet", unique=True)
```
> @stephanie-wang That was another regression I noticed, adding a `unique=True` argument to `get_log_file_names` [here](https://github.com/ray-project/ray/blob/5881417ec4a993194ba6d64b156ad6b5ffbca97c/python/ray/node.py#L650) should do the trick:
> 
> ```
> raylet_out_name, raylet_err_name = self.get_log_file_names("raylet", unique=True)
> ```

Ah, nice find!
Nice catch, I may have made this same mistake in a few places. @clarkzinzow do you want to submit a PR for this? If not, I'll try to get around to it in a week or so. 
@wuisawesome Sure I can submit a PR for that, I'm already in this code for [another PR](https://github.com/ray-project/ray/pull/9547) so it's not much of a context switch.

Resolving patch:
diff --git a/python/ray/node.py b/python/ray/node.py
--- a/python/ray/node.py
+++ b/python/ray/node.py
@@ -647,7 +647,8 @@ def start_raylet(self, use_valgrind=False, use_profiler=False):
             use_profiler (bool): True if we should start the process in the
                 valgrind profiler.
         """
-        raylet_out_name, raylet_err_name = self.get_log_file_names("raylet")
+        raylet_out_name, raylet_err_name = self.get_log_file_names(
+            "raylet", unique=True)
         stdout_file, stderr_file = (open_log(raylet_out_name),
                                     open_log(raylet_err_name))
         process_info = ray.services.start_raylet(


Test patch:
Repository: ray-project/ray

Issue:
[rllib] Train script overwrites framework field in experiment config argument
[rllib]

### What is the problem?

Framework setting passed to training script via config argument is ignored, because training script uses separate arguments for this field (`--torch`, `--eager`). If none of these args is passed to the script the framework is set to `tf`, regardless of options passed in `--config` argument.

### Expected behavior

Script should use default value for `framework` field only if it is not already present in `config` dict (it can be passed as cmd argument or be loaded from file).

### Reproduction (REQUIRED)
Run `rllib train --run DQN --env CartPole-v0 --config "{\"framework\": \"tfe\"}"` - script uses `tf` instead of `tfe`

- [x] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:

Resolving patch:
diff --git a/rllib/train.py b/rllib/train.py
--- a/rllib/train.py
+++ b/rllib/train.py
@@ -180,22 +180,23 @@ def run(args, parser):
             parser.error("the following arguments are required: --run")
         if not exp.get("env") and not exp.get("config", {}).get("env"):
             parser.error("the following arguments are required: --env")
-        if args.eager:
-            exp["config"]["framework"] = "tfe"
-        elif args.torch:
+
+        if args.torch:
             exp["config"]["framework"] = "torch"
-        else:
-            exp["config"]["framework"] = "tf"
+        elif args.eager:
+            exp["config"]["framework"] = "tfe"
+
+        if args.trace:
+            if exp["config"]["framework"] not in ["tf2", "tfe"]:
+                raise ValueError("Must enable --eager to enable tracing.")
+            exp["config"]["eager_tracing"] = True
+
         if args.v:
             exp["config"]["log_level"] = "INFO"
             verbose = 2
         if args.vv:
             exp["config"]["log_level"] = "DEBUG"
             verbose = 3
-        if args.trace:
-            if exp["config"]["framework"] != "tfe":
-                raise ValueError("Must enable --eager to enable tracing.")
-            exp["config"]["eager_tracing"] = True
 
     if args.ray_num_nodes:
         cluster = Cluster()


Test patch:
Repository: ray-project/ray

Issue:
[dashboard] When autoscaler adds/removes worker, dashboard crashes
<!--Please include [tune], [rllib], [autoscaler] etc. in the issue title if relevant-->

### What is the problem?
This is originally reported by Pieterjan for 0.8.7 but also appears to be in issue in the latest nightly. The issue would appear to occur across 

### Reproduction (REQUIRED)
This is not a script, but given I will author the fix, I think this is sufficient:
- Start a cluster with autoscaling enabled
- Increase workload so that an additional worker or workers are spawned
- When one of those workers is removed, their log entries still exist on the backend
- The following stack trace occurs and crashes the dashboard: `/ray/dashboard/node_stats.py", line 63, in _insert_log_counts
    self._node_stats[hostname]["log_count"] = logs_by_pid` 

- [ ] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:

Resolving patch:
diff --git a/python/ray/dashboard/node_stats.py b/python/ray/dashboard/node_stats.py
--- a/python/ray/dashboard/node_stats.py
+++ b/python/ray/dashboard/node_stats.py
@@ -60,14 +60,22 @@ def __init__(self, redis_address, redis_password=None):
     def _insert_log_counts(self):
         for ip, logs_by_pid in self._logs.items():
             hostname = self._ip_to_hostname[ip]
-            logs_by_pid = {pid: len(logs) for pid, logs in logs_by_pid.items()}
-            self._node_stats[hostname]["log_count"] = logs_by_pid
+            if hostname in self._node_stats:
+                logs_by_pid = {
+                    pid: len(logs)
+                    for pid, logs in logs_by_pid.items()
+                }
+                self._node_stats[hostname]["log_count"] = logs_by_pid
 
     def _insert_error_counts(self):
         for ip, errs_by_pid in self._errors.items():
             hostname = self._ip_to_hostname[ip]
-            errs_by_pid = {pid: len(errs) for pid, errs in errs_by_pid.items()}
-            self._node_stats[hostname]["error_count"] = errs_by_pid
+            if hostname in self._node_stats:
+                errs_by_pid = {
+                    pid: len(errs)
+                    for pid, errs in errs_by_pid.items()
+                }
+                self._node_stats[hostname]["error_count"] = errs_by_pid
 
     def _purge_outdated_stats(self):
         def current(then, now):


Test patch:
Repository: ray-project/ray

Issue:
[rllib] SAC and DDPG policy can't do `grad_clip`
<!--Please include [tune], [rllib], [autoscaler] etc. in the issue title if relevant-->

### What is the problem?

*Ray version and other system information (Python version, TensorFlow version, OS):*
Ray: 0.8.7
Python: 3.7
Tensorflow: 2.2.0
OS: Manjaro x86_64


### Reproduction (REQUIRED)
Please provide a script that can be run to reproduce the issue. The script should have **no external library dependencies** (i.e., use fake or mock data / environments):
````py
from ray import tune
from ray.rllib.agents.sac import SACTrainer
from ray.rllib.agents.ddpg import DDPGTrainer

tune.run(DDPGTrainer, config={"env": "CartPole-v0","grad_clip":40.0})
# tune.run(SACTrainer, config={"env": "CartPole-v0","grad_clip":40.0})
````
This [PR](https://github.com/ray-project/ray/pull/10499) will fix this issue.

- [x] I have verified my script runs in a clean environment and reproduces the issue.
- [x] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:

Resolving patch:
diff --git a/rllib/agents/ddpg/ddpg_tf_policy.py b/rllib/agents/ddpg/ddpg_tf_policy.py
--- a/rllib/agents/ddpg/ddpg_tf_policy.py
+++ b/rllib/agents/ddpg/ddpg_tf_policy.py
@@ -1,4 +1,5 @@
 from gym.spaces import Box
+from functools import partial
 import logging
 import numpy as np
 
@@ -290,7 +291,8 @@ def gradients_fn(policy, optimizer, loss):
 
     # Clip if necessary.
     if policy.config["grad_clip"]:
-        clip_func = tf.clip_by_norm
+        clip_func = partial(
+            tf.clip_by_norm, clip_norm=policy.config["grad_clip"])
     else:
         clip_func = tf.identity
 
diff --git a/rllib/agents/sac/sac_tf_policy.py b/rllib/agents/sac/sac_tf_policy.py
--- a/rllib/agents/sac/sac_tf_policy.py
+++ b/rllib/agents/sac/sac_tf_policy.py
@@ -1,4 +1,5 @@
 from gym.spaces import Box, Discrete
+from functools import partial
 import logging
 
 import ray
@@ -322,7 +323,8 @@ def gradients_fn(policy, optimizer, loss):
 
     # Clip if necessary.
     if policy.config["grad_clip"]:
-        clip_func = tf.clip_by_norm
+        clip_func = partial(
+            tf.clip_by_norm, clip_norm=policy.config["grad_clip"])
     else:
         clip_func = tf.identity
 


Test patch:
Repository: ray-project/ray

Issue:
[dask-on-ray] Set task name to Dask key at task submission time.
<!--Please include [tune], [rllib], [autoscaler] etc. in the issue title if relevant-->

### Describe your feature request

To provide better dashboard, metrics, and debugging UX, we should set the task name to the Dask key. The Dask-Ray scheduler [uses a single remote function](https://github.com/ray-project/ray/blob/ef18893fb50c97c2c0d9d7b8b951455683f89635/python/ray/experimental/dask/scheduler.py#L208-L233) for executing all Dask tasks; Dask task functions are all dynamically composed (via partial application and the like), and using this single remote function allows us to cut down on function exporting overhead. However, this means that the default task name (derived from the function descriptor) is the same for every Dask task, which hurts dashboard and debugging UX considerably. By setting the task name to the Dask key at task submission time, we'll be able to attach meaningful semantics to each task that will be displayed in the dashboard and included in task spec log records.

### Blockers

Blocked by https://github.com/ray-project/ray/issues/10371.


Discussion:

Resolving patch:
diff --git a/python/ray/experimental/dask/scheduler.py b/python/ray/experimental/dask/scheduler.py
--- a/python/ray/experimental/dask/scheduler.py
+++ b/python/ray/experimental/dask/scheduler.py
@@ -196,7 +196,8 @@ def _rayify_task(task, key, deps):
         # Ray properly tracks the object dependencies between Ray tasks.
         object_refs, repack = unpack_object_refs(args, deps)
         # Submit the task using a wrapper function.
-        return dask_task_wrapper.remote(func, repack, *object_refs)
+        return dask_task_wrapper.options(name=f"dask:{key!s}").remote(
+            func, repack, *object_refs)
     elif not ishashable(task):
         return task
     elif task in deps:


Test patch:
Repository: ray-project/ray

Issue:
[autoscaler] Rsync breaks when `target` or `source` is None
<!--Please include [tune], [rllib], [autoscaler] etc. in the issue title if relevant-->

### What is the problem?
When `rsync` is run with `source` and `target` set to `None` and `file_mounts` are included in the YAML, ray fails with
```
None type is not iterable
```

Introduced by #10633 

*Ray version and other system information (Python version, TensorFlow version, OS):*

### Reproduction (REQUIRED)
Please provide a script that can be run to reproduce the issue. The script should have **no external library dependencies** (i.e., use fake or mock data / environments):

If we cannot run your script, we cannot fix your issue.

- [ ] I have verified my script runs in a clean environment and reproduces the issue.
- [ ] I have verified the issue also occurs with the [latest wheels](https://docs.ray.io/en/latest/installation.html).



Discussion:

Resolving patch:
diff --git a/python/ray/autoscaler/commands.py b/python/ray/autoscaler/commands.py
--- a/python/ray/autoscaler/commands.py
+++ b/python/ray/autoscaler/commands.py
@@ -959,10 +959,11 @@ def rsync(config_file: str,
     config = _bootstrap_config(config, no_config_cache=no_config_cache)
 
     is_file_mount = False
-    for remote_mount in config.get("file_mounts", {}).keys():
-        if remote_mount in (source if down else target):
-            is_file_mount = True
-            break
+    if source and target:
+        for remote_mount in config.get("file_mounts", {}).keys():
+            if (source if down else target).startswith(remote_mount):
+                is_file_mount = True
+                break
 
     provider = get_node_provider(config["provider"], config["cluster_name"])
     try:


Test patch: