Repository: microsoft/DeepSpeed

Issue:
[BUG] "Setting ds_accelerator" uses `print` not `logger`
**Describe the bug**
When `accelerate` tries to run any command while `DeepSpeed` is available, we get `"Setting ds_accelerator ..."`, which while not explicitly bad, is making a test fail due to a non-clean CLI print when non-deepspeed code is used (we have a check for `is_deepspeed_available()` which is probably triggering this.

As a nice QOL, it would be good to have `get_accelerator` (https://github.com/microsoft/DeepSpeed/blob/master/accelerator/real_accelerator.py#L102) and `set_accelerator` (https://github.com/microsoft/DeepSpeed/blob/master/accelerator/real_accelerator.py#L109) use the already-existing logger in the framework if possible, so that we can disable these annoying prints when they're not needed :)

**To Reproduce**
Steps to reproduce the behavior:
1. `pip install accelerate deepspeed -U`
2. Create a file with:
```python
from accelerate.commands.tpu import tpu_command_parser, tpu_command_launcher

parser = tpu_command_parser()
args = parser.parse_args([
    "--config_file", "tests/test_configs/latest.yaml",
    "--install_accelerate",
    "--debug"
])

tpu_command_launcher(args)
```
3. Run `python {my_file_name.py}`
4. Should print:
```bash
Setting ds_accelerator to cuda (auto detect)
Running gcloud compute tpus tpu-vm ssh test-tpu --zone us-central1-a --command cd /usr/share; pip install accelerate -U; echo "hello world"; echo "this is a second command" --worker all
```

**Expected behavior**

A configurable option to silence these print statements by having them run through the logging system instead, perhaps as an `info` or as a `debug`. 


Discussion:
@muellerzr, thanks for reporting this error. This is definitely an oversight. Are you able to submit a PR that uses logger instead of print?
@tjruwase sadly don't have the bandwidth/familiarity with the code right now to do so, would appreciate it if someone else could pick it up 🙏 
No worries. Thanks for reporting the error. I will link a PR asap. 
By the way, are you actually running with tpus? The accelerator abstraction is a fairly new feature to make it easier to run DeepSpeed on arbitrary hardware, and so we appreciate any feedback. Thanks! 
@tjruwase nope, not with DeepSpeed yet (@pacman100 on that one with Accelerate?) it's on a GPU runner, just our CLI picks up on it due to import (which was also a great way to know that something weird was happening 😄 )

Resolving patch:
diff --git a/accelerator/real_accelerator.py b/accelerator/real_accelerator.py
--- a/accelerator/real_accelerator.py
+++ b/accelerator/real_accelerator.py
@@ -4,6 +4,13 @@
 # DeepSpeed Team
 import os
 
+try:
+    # Importing logger currently requires that torch is installed, hence the try...except
+    # TODO: Remove logger dependency on torch.
+    from deepspeed.utils import logger as accel_logger
+except ImportError as e:
+    accel_logger = None
+
 try:
     from accelerator.abstract_accelerator import DeepSpeedAccelerator as dsa1
 except ImportError as e:
@@ -99,14 +106,16 @@ def get_accelerator():
         # XPU_Accelerator is already imported in detection stage
         ds_accelerator = XPU_Accelerator()
     _validate_accelerator(ds_accelerator)
-    print(f"Setting ds_accelerator to {ds_accelerator._name} ({ds_set_method})")
+    if accel_logger is not None:
+        accel_logger.info(f"Setting ds_accelerator to {ds_accelerator._name} ({ds_set_method})")
     return ds_accelerator
 
 
 def set_accelerator(accel_obj):
     global ds_accelerator
     _validate_accelerator(accel_obj)
-    print(f"Setting ds_accelerator to {accel_obj._name} (model specified)")
+    if accel_logger is not None:
+        accel_logger.info(f"Setting ds_accelerator to {accel_obj._name} (model specified)")
     ds_accelerator = accel_obj
 
 
@@ -114,10 +123,10 @@ def set_accelerator(accel_obj):
 -----------[code] test_get.py -----------
 from deepspeed.accelerator import get_accelerator
 my_accelerator = get_accelerator()
-print(f'{my_accelerator._name=}')
-print(f'{my_accelerator._communication_backend=}')
-print(f'{my_accelerator.HalfTensor().device=}')
-print(f'{my_accelerator.total_memory()=}')
+logger.info(f'{my_accelerator._name=}')
+logger.info(f'{my_accelerator._communication_backend=}')
+logger.info(f'{my_accelerator.HalfTensor().device=}')
+logger.info(f'{my_accelerator.total_memory()=}')
 -----------[code] test_get.py -----------
 
 ---[output] python test_get.py---------
@@ -131,16 +140,16 @@ def set_accelerator(accel_obj):
 -----------[code] test_set.py -----------
 from deepspeed.accelerator.cuda_accelerator import CUDA_Accelerator
 cu_accel = CUDA_Accelerator()
-print(f'{id(cu_accel)=}')
+logger.info(f'{id(cu_accel)=}')
 from deepspeed.accelerator import set_accelerator, get_accelerator
 set_accelerator(cu_accel)
 
 my_accelerator = get_accelerator()
-print(f'{id(my_accelerator)=}')
-print(f'{my_accelerator._name=}')
-print(f'{my_accelerator._communication_backend=}')
-print(f'{my_accelerator.HalfTensor().device=}')
-print(f'{my_accelerator.total_memory()=}')
+logger.info(f'{id(my_accelerator)=}')
+logger.info(f'{my_accelerator._name=}')
+logger.info(f'{my_accelerator._communication_backend=}')
+logger.info(f'{my_accelerator.HalfTensor().device=}')
+logger.info(f'{my_accelerator.total_memory()=}')
 -----------[code] test_set.py -----------
 
 


Test patch: