Repository: ray-project/ray
Pull request: Fix python executable path for ray nsight configuration
## Why are these changes needed?

Not all systems are guaranteed to have "python" be the way that the python interpreter is invoked on the command line. The proper way to find python is to use `sys.executable`. This hardcoded "python" was breaking workflows within my Ubuntu 22.04 training environment.

## Related issue number

Closes #53597 

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [x] I've included any doc changes needed for https://docs.ray.io/en/master/.
    - [ ] I've added any new APIs to the API Reference. For example, if I added a
           method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [x] This PR is not tested :(

Problem statement:
Nsight GPU profiling unable to run on systems that don't have a "python" on the PATH
### What happened + What you expected to happen

Ever since the end-of-life of Python 2, many Linux distributions (including Ubuntu) do not have a command by default named "python". Oftentimes python has to be invoked with "python3" since "python" is historically an alias to Python 2 on systems.

When using the ray `runtime_env` with `"nsight": "default"` the program fails if there is no existing "python" program on the machine.

On my machine I can get this to work only after I run `sudo ln -s /usr/bin/python3 /usr/bin/python`

### Versions / Dependencies

OS: Ubuntu 22.04

Python: 3.10

Ray: 2.30.0

### Reproduction script

# This is the code directly from the Ray docs

import torch
import ray

ray.init()

@ray.remote(num_gpus=1, runtime_env={ "nsight": "default"})
class RayActor:
    def run():
    a = torch.tensor([1.0, 2.0, 3.0]).cuda()
    b = torch.tensor([4.0, 5.0, 6.0]).cuda()
    c = a * b

    print("Result on GPU:", c)

ray_actor = RayActor.remote()
ray.get(ray_actor.run.remote())

### Issue Severity

Medium: It is a significant difficulty but I can work around it.

Hints from issue discussion:

Commit message: Fix python executable path for ray nsight configuration

Signed-off-by: Shishir Bhat <shishirvbhat@gmail.com>

Patch to review:
diff --git a/python/ray/_private/runtime_env/nsight.py b/python/ray/_private/runtime_env/nsight.py
index 91b012db347b5..1cac735d2963d 100644
--- a/python/ray/_private/runtime_env/nsight.py
+++ b/python/ray/_private/runtime_env/nsight.py
@@ -71,7 +71,8 @@ async def _check_nsight_script(
         nsight_config_copy["o"] = str(Path(self._nsight_dir) / "empty")
         nsight_cmd = parse_nsight_config(nsight_config_copy)
         try:
-            nsight_cmd = nsight_cmd + ["python", "-c", '""']
+            python_executable = sys.executable or "python"
+            nsight_cmd = nsight_cmd + [python_executable, "-c", '""']
             process = await asyncio.create_subprocess_exec(
                 *nsight_cmd,
                 stdout=subprocess.PIPE,


Code review comments:
diff_hunk: @@ -71,7 +71,8 @@ async def _check_nsight_script(
         nsight_config_copy["o"] = str(Path(self._nsight_dir) / "empty")
         nsight_cmd = parse_nsight_config(nsight_config_copy)
         try:
-            nsight_cmd = nsight_cmd + ["python", "-c", '""']
+            python_executable = sys.executable or "python"
path: python/ray/_private/runtime_env/nsight.py
text: @user1:
We can assume that `sys.executable` is available, since we do so in many other places. Why don't you update this to
```
nsight_cmd = nsight_cmd + [sys.executable, "-c", '""']
```
to make it simpler?

I believe it would only not be available if the script is executed with a statically or dynamically linked python library, which is a very exotic setting (and even if a user uses this, they could easily set sys.executable to a proper interpreter before they invoke the python script).

@author:
Thanks for the quick review @user1 . Simplified it. Wasn't sure if there was a type checker in the repo that would get mad at the line since `sys.executable` is allowed to be `None`

@user1:
Ah makes sense -- I don't think that's a problem, the official type stub for sys.executable seems to be str: https://github.com/python/typeshed/blob/567b488fc28978642bfeb70b29f11453b0ff124c/stdlib/sys/__init__.pyi#L31