Repository: pytorch/benchmark
Issue: python install.py pollutes global python package environment
Root cause: External Dependency Breakage

I added a print statement here

```diff
diff --git a/torchbenchmark/__init__.py b/torchbenchmark/__init__.py
index 72266207..6ce5a9d1 100644
--- a/torchbenchmark/__init__.py
+++ b/torchbenchmark/__init__.py
@@ -115,6 +115,7 @@ def setup(models: List[str] = [], verbose: bool = True, continue_on_fail: bool =
             print("SKIP - No install.py is found")
         elif success:
             print("OK")
+            subprocess.run('ls -la /opt/conda/lib/python3.8/site-packages | grep -i tools', shell=True, check=False)
         else:
             print("FAIL")
             try:
```

As I was running `python install.py` ...
```
...
00:00:40 running setup for /workspace/torch-benchmark/benchmark/torchbenchmark/models/demucs...drwxr-xr-x 23 root root     4096 Jun 16 09:11 fontTools
00:00:40 drwxr-xr-x  2 root root     4096 Jun 16 09:11 fonttools-4.33.3.dist-info
00:00:40 drwxr-xr-x  3 root root     4096 Jun 16 09:11 pycocotools
00:00:40 drwxr-xr-x  2 root root     4096 Jun 16 09:11 pycocotools-2.0+nv0.4.0.dist-info
00:00:40 drwxr-xr-x  8 root root     4096 Jun 16 07:36 setuptools
00:00:40 drwxr-xr-x  2 root root     4096 Jun 16 07:36 setuptools-61.2.0-py3.8.egg-info
00:00:40 OK
00:00:40 running setup for /workspace/torch-benchmark/benchmark/torchbenchmark/models/densenet121...drwxr-xr-x 23 root root     4096 Jun 16 09:11 fontTools
00:00:40 drwxr-xr-x  2 root root     4096 Jun 16 09:11 fonttools-4.33.3.dist-info
00:00:40 drwxr-xr-x  3 root root     4096 Jun 16 09:11 pycocotools
00:00:40 drwxr-xr-x  2 root root     4096 Jun 16 09:11 pycocotools-2.0+nv0.4.0.dist-info
00:00:40 drwxr-xr-x  8 root root     4096 Jun 16 07:36 setuptools
00:00:40 drwxr-xr-x  2 root root     4096 Jun 16 07:36 setuptools-61.2.0-py3.8.egg-info
00:00:40 OK
00:02:16 running setup for /workspace/torch-benchmark/benchmark/torchbenchmark/models/detectron2_fasterrcnn_r_101_c4...drwxr-xr-x  3 root root     4096 Jun 18 06:56 cachetools
00:02:16 drwxr-xr-x  2 root root     4096 Jun 18 06:56 cachetools-5.2.0.dist-info
00:02:16 drwxr-xr-x 23 root root     4096 Jun 16 09:11 fontTools
00:02:16 drwxr-xr-x  2 root root     4096 Jun 16 09:11 fonttools-4.33.3.dist-info
00:02:16 drwxr-xr-x  3 root root     4096 Jun 18 06:56 pycocotools
00:02:16 drwxr-xr-x  2 root root     4096 Jun 18 06:56 pycocotools-2.0.4.dist-info
00:02:16 drwxr-xr-x  8 root root     4096 Jun 16 07:36 setuptools
00:02:16 drwxr-xr-x  2 root root     4096 Jun 16 07:36 setuptools-61.2.0-py3.8.egg-info
00:02:16 drwxr-xr-x  3 root root     4096 Jun 18 06:56 tools
00:02:16 OK
00:02:36 running setup for /workspace/torch-benchmark/benchmark/torchbenchmark/models/detectron2_fasterrcnn_r_101_dc5...drwxr-xr-x  3 root root     4096 Jun 18 06:56 cachetools
00:02:36 drwxr-xr-x  2 root root     4096 Jun 18 06:56 cachetools-5.2.0.dist-info
00:02:36 drwxr-xr-x 23 root root     4096 Jun 16 09:11 fontTools
00:02:36 drwxr-xr-x  2 root root     4096 Jun 16 09:11 fonttools-4.33.3.dist-info
00:02:36 drwxr-xr-x  3 root root     4096 Jun 18 06:56 pycocotools
00:02:36 drwxr-xr-x  2 root root     4096 Jun 18 06:56 pycocotools-2.0.4.dist-info
00:02:36 drwxr-xr-x  8 root root     4096 Jun 16 07:36 setuptools
00:02:36 drwxr-xr-x  2 root root     4096 Jun 16 07:36 setuptools-61.2.0-py3.8.egg-info
00:02:36 drwxr-xr-x  3 root root     4096 Jun 18 06:56 tools
...
```

After `detectron2_fasterrcnn_r_101_c4` model is installed, there is a `tools` folder installed in the global conda package list. I checked the contents of this path, it's exactly from https://github.com/facebookresearch/detectron2/tree/main/tools. 

This is causing python import failures, for example

https://github.com/pytorch/pytorch/blob/e10cbe388042e2dc9b5919c37d91aee7a06cc858/test/run_test.py#L32-L41

```python
try:
    # using tools/ to optimize test run.
    sys.path.append(str(REPO_ROOT))
    from tools.testing.test_selections import (
        export_S3_test_times,
        get_shard_based_on_S3,
        get_reordered_tests,
        get_test_case_configs,
    )
    HAVE_TEST_SELECTION_TOOLS = True
```

This code will try to import `tools` from conda global path instead of pytorch local path, which causes import failure.

cc @xuzhao9 @malfet @ptrblck

Looks like this is caused by upstream bug: https://github.com/facebookresearch/detectron2/issues/3934

Workaround this problem in https://github.com/pytorch/benchmark/pull/966.
It first check if the package `tools` exists in the current python environment, if so, delete it with `shutil.rmtree()`.

Closed by https://github.com/pytorch/benchmark/pull/979 (workaround)