Repository: microsoft/DeepSpeed

Issue:
[BUG] `pip install deepspeed==0.13.0` fails
I commented on the [release/build commit](https://github.com/microsoft/DeepSpeed/commit/1c8b8f30cbb0515693cff70cb4784c675251e295#comments), but reposting as an issue for better visibility. 

Installing fails on my end, when trying to install `deepspeed==0.13.0` with
```bash
conda create -p ./.venv python=3.9.18
conda activate ./.venv
pip install deepspeed==0.13.0
```

(see error below)

Solutions: I think `torch` should either become a build requirement for `deepspeed` or the build code should be adjusted to prevent this error. This build time error seems to have been introduced in one of the commits in v0.13.0. Then, I think https://github.com/microsoft/DeepSpeed/commit/1c8b8f30cbb0515693cff70cb4784c675251e295 can be reverted.

### Error
```
Collecting deepspeed==0.13.0
  Downloading deepspeed-0.13.0.tar.gz (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 22.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Traceback (most recent call last):
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/builder.py", line 14, in <module>
          from op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
      ImportError: cannot import name 'TORCH_MAJOR' from 'op_builder.builder' (/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/builder.py)
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/setup.py", line 37, in <module>
          from op_builder import get_default_compute_capabilities, OpBuilder
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/__init__.py", line 48, in <module>
          module = importlib.import_module(f".{module_name}", package=op_builder_dir)
        File ".venv/lib/python3.9/importlib/__init__.py", line 127, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/__init__.py", line 6, in <module>
          from .cpu_adam import CPUAdamBuilder
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/cpu_adam.py", line 6, in <module>
          from .builder import SYCLOpBuilder
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/builder.py", line 16, in <module>
          from deepspeed.ops.op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/deepspeed/__init__.py", line 10, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [WARNING] Unable to import torch, pre-compiling ops will be disabled. Please visit https://pytorch.org/ to see how to properly install torch on your system.
       [WARNING]  unable to import torch, please install it if you want to pre-compile any deepspeed ops.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
```


Discussion:
cc @mrwyattii @jeffra 
Hi @apoorvkh while in the past we have not required torch to be installed before installing DeepSpeed, it has always been required to run DeepSpeed. This "bug" is from a recent PR: #4547

It is problematic adding torch as a requirement to the project because pip will often install the CPU version rather than CUDA/ROCm versions.

We are discussing internally what the proper action is, but I suspect we will restore the previous behavior of not requiring torch at install time and do a 0.13.1 patch release with this update.
Understood, that would be great -- thank you!
This issue is caused by these two piece of reflection codes.  The problem is the reflection code should not intend to iterate into directories belongs to other accelerator.

https://github.com/microsoft/DeepSpeed/blob/master/op_builder/__init__.py#L46
https://github.com/microsoft/DeepSpeed/blob/master/op_builder/all_ops.py#L22

There are two possible fixes, one is to skip directories belongs to accelerators with code like:
`if module_name not in ['cpu', 'hpu', 'mps', 'npu', 'xpu']:`
Another possible fix which is more graceful should be move CUDA OpBuilders into `cuda/` directory in op_builder but his might need more global code change.


Resolving patch:
diff --git a/op_builder/xpu/async_io.py b/op_builder/xpu/async_io.py
--- a/op_builder/xpu/async_io.py
+++ b/op_builder/xpu/async_io.py
@@ -5,7 +5,6 @@
 
 import distutils.spawn
 import subprocess
-import torch
 
 from .builder import OpBuilder
 
@@ -33,6 +32,7 @@ def include_paths(self):
         return ['csrc/aio/py_lib', 'csrc/aio/common']
 
     def cxx_args(self):
+        import torch
         # -O0 for improved debugging, since performance is bound by I/O
         CPU_ARCH = self.cpu_arch()
         SIMD_WIDTH = self.simd_width()
diff --git a/op_builder/xpu/builder.py b/op_builder/xpu/builder.py
--- a/op_builder/xpu/builder.py
+++ b/op_builder/xpu/builder.py
@@ -11,9 +11,9 @@
     # is op_builder from deepspeed or a 3p version? this should only succeed if it's deepspeed
     # if successful this also means we're doing a local install and not JIT compile path
     from op_builder import __deepspeed__  # noqa: F401 # type: ignore
-    from op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
+    from op_builder.builder import OpBuilder
 except ImportError:
-    from deepspeed.ops.op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
+    from deepspeed.ops.op_builder.builder import OpBuilder
 
 
 class SYCLOpBuilder(OpBuilder):
@@ -35,6 +35,10 @@ def builder(self):
         return dpcpp_ext
 
     def version_dependent_macros(self):
+        try:
+            from op_builder.builder import TORCH_MAJOR, TORCH_MINOR
+        except ImportError:
+            from deepspeed.ops.op_builder.builder import TORCH_MAJOR, TORCH_MINOR
         # Fix from apex that might be relevant for us as well, related to https://github.com/NVIDIA/apex/issues/456
         version_ge_1_1 = []
         if (TORCH_MAJOR > 1) or (TORCH_MAJOR == 1 and TORCH_MINOR > 0):


Test patch: