Repository: microsoft/DeepSpeed

Issue:
[BUG] DeepSpeed 0.7.0 breaks the Accelerate CI due to pip installation
**Describe the bug**
For Accelerate we have deepspeed as an extra that gets installed during tests. As of 0.7.0 DeepSpeed is unable to be installed if pytorch is not already installed on the system, so doing `pip install -e .[testing]` (which installs torch and deepspeed at once) fails due to a compilation error on DeepSpeed's side. 

**To Reproduce**
Steps to reproduce the behavior:
1. In a new environment:
```bash
git clone https://github.com/huggingface/accelerate
cd accelerate
pip install -e .[testing]
```
It won't get past the pip install

**Expected behavior**
Should install just fine

**System info (please complete the following information):**
 - OS: Ubuntu 20.04
 - GPU count and types: None
 - Interconnects (if applicable) [e.g., two machines connected with 100 Gbps IB]
 - Python version: 3.8.13
 - Any other relevant info about your setup

**Launcher context**
Are you launching your experiment with the `deepspeed` launcher, MPI, or something else?
No

**Docker context**
Are you using a specific docker image that you can share?
No

**Additional context**
Add any other context about the problem here.

Full installation trace:
```bash
(deepspeed) zach@workstation:~/accelerate/accelerate$ pip install -e .[testing]
Obtaining file:///home/zach/accelerate/accelerate
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.17 in /home/zach/.local/lib/python3.8/site-packages (from accelerate==0.12.0.dev0) (1.23.0)
Requirement already satisfied: packaging>=20.0 in /home/zach/.local/lib/python3.8/site-packages (from accelerate==0.12.0.dev0) (21.3)
Requirement already satisfied: psutil in /home/zach/.local/lib/python3.8/site-packages (from accelerate==0.12.0.dev0) (5.9.1)
Collecting pyyaml
  Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting torch>=1.4.0
  Downloading torch-1.12.0-cp38-cp38-manylinux1_x86_64.whl (776.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 776.3/776.3 MB 4.6 MB/s eta 0:00:00
Collecting deepspeed
  Downloading deepspeed-0.7.0.tar.gz (629 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 630.0/630.0 kB 9.2 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [12 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-022a34y9/deepspeed_68937ff3b2bb4dbdbd7449fa5f5103b6/setup.py", line 33, in <module>
          from op_builder import ALL_OPS, get_default_compute_capabilities, OpBuilder
        File "/tmp/pip-install-022a34y9/deepspeed_68937ff3b2bb4dbdbd7449fa5f5103b6/op_builder/__init__.py", line 13, in <module>
          from .transformer_inference import InferenceBuilder
        File "/tmp/pip-install-022a34y9/deepspeed_68937ff3b2bb4dbdbd7449fa5f5103b6/op_builder/transformer_inference.py", line 1, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [WARNING] Unable to import torch, pre-compiling ops will be disabled. Please visit https://pytorch.org/ to see how to properly install torch on your system.
       [WARNING]  unable to import torch, please install it if you want to pre-compile any deepspeed ops.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
```


Discussion:

Resolving patch:
diff --git a/op_builder/transformer_inference.py b/op_builder/transformer_inference.py
--- a/op_builder/transformer_inference.py
+++ b/op_builder/transformer_inference.py
@@ -1,4 +1,3 @@
-import torch
 from .builder import CUDAOpBuilder, installed_cuda_version
 
 
@@ -14,6 +13,13 @@ def absolute_name(self):
         return f'deepspeed.ops.transformer.inference.{self.NAME}_op'
 
     def is_compatible(self, verbose=True):
+        try:
+            import torch
+        except ImportError:
+            self.warning(
+                "Please install torch if trying to pre-compile inference kernels")
+            return False
+
         cuda_okay = True
         if not self.is_rocm_pytorch() and torch.cuda.is_available():
             sys_cuda_major, _ = installed_cuda_version()


Test patch: