Repository: pytorch
Issue: Get https://github.com/pytorch/benchmark working
Labels: module: windows,triaged

### Description
- The https://github.com/pytorch/benchmark seems to be the official PyTorch performance benchmark running on the CI.
- This is a subtask of https://github.com/microsoft/ARM-on-Windows-Ecosystem/issues/187

### Acceptance Criteria
- We can compare results of https://github.com/pytorch/benchmark on Windows and WSL.

### Documentation

Steps to make this benchmark working:
- Enable long file names policy [Make Windows 11 Accept File Paths over 260 Characters (thegeekpage.com)](https://thegeekpage.com/make-windows-11-accept-file-paths-over-260-characters/) 
  - Still an issue [Add long path support for Windows [Approach #2] by majaeger · Pull Request #2056 · ninja-build/ninja (github.com)](https://github.com/ninja-build/ninja/pull/2056) 
- `conda create -n pytorch-benchmark-py3.8 python=3.8`
- `conda activate pytorch-benchmark-py3.8`
- `conda install -y -c pytorch pytorch torchvision torchtext`
- `conda install git-lfs pyyaml`
- `git clone https://github.com/pytorch/benchmark`
- Apply the following patch:
  ```
  diff --git a/torchbenchmark/__init__.py b/torchbenchmark/__init__.py
  index 897dbe23..92c1aca9 100644
  --- a/torchbenchmark/__init__.py
  +++ b/torchbenchmark/__init__.py
  @@ -43,10 +43,11 @@ internal_model_dir = "fb"
   install_file = 'install.py'
 
 
  -def _test_https(test_url: str = 'https://github.com', timeout: float = 0.5) -> bool:
  +def _test_https(test_url: str = 'https://github.com', timeout: float = 5.0) -> bool:
       try:
           request.urlopen(test_url, timeout=timeout)
  -    except OSError:
  +    except OSError as e:
  +        print(e)
           return False
       return True
 
  @@ -57,7 +58,7 @@ def _install_deps(model_path: str, verbose: bool = True) -> Tuple[bool, Any]:
           [sys.executable, install_file],
       ]
       run_env = os.environ.copy()
  -    run_env["PYTHONPATH"] = this_dir.parent
  +    run_env["PYTHONPATH"] = str(this_dir.parent)
       run_kwargs = {
           'cwd': model_path,
           'check': True,
  @@ -88,7 +89,10 @@ def _install_deps(model_path: str, verbose: bool = True) -> Tuple[bool, Any]:
           return (False, e, io.FileIO(stdout_fpath, mode="r").read().decode())
       finally:
           del output_buffer
  -        os.remove(stdout_fpath)
  +        try:
  +            os.remove(stdout_fpath)
  +        except:
  +            pass
 
       return (True, None, None)
 
  diff --git a/torchbenchmark/models/fambench_xlmr/requirements.txt 
  b/torchbenchmark/models/fambench_xlmr/requirements.txt
  index da75cfcd..06cb9fef 100644
  --- a/torchbenchmark/models/fambench_xlmr/requirements.txt
  +++ b/torchbenchmark/models/fambench_xlmr/requirements.txt
  @@ -1,8 +1,7 @@
   sacrebleu
   bitarray
   # pin fairseq version
  -fairseq==0.10.2
  +fairseq==0.10.0
   omegaconf==2.1.1
   hydra-core==1.1.2
   sentencepiece
  -xformers
  diff --git a/torchbenchmark/models/opacus_cifar10/requirements.txt 
  b/torchbenchmark/models/opacus_cifar10/requirements.txt
  index 106f0456..6a68fd02 100644
  --- a/torchbenchmark/models/opacus_cifar10/requirements.txt
  +++ b/torchbenchmark/models/opacus_cifar10/requirements.txt
  @@ -1,3 +1,2 @@
  -git+https://github.com/pytorch/functorch.git
  -# must include the fix https://github.com/pytorch/opacus/pull/426
  +functorch
   opacus>=1.1.2
  diff --git a/torchbenchmark/util/framework/detectron2/requirements.txt 
  b/torchbenchmark/util/framework/detectron2/requirements.txt
  index a7f6a571..3c17e0b6 100644
  --- a/torchbenchmark/util/framework/detectron2/requirements.txt
  +++ b/torchbenchmark/util/framework/detectron2/requirements.txt
  @@ -1,3 +1,2 @@
  -git+https://github.com/facebookresearch/detectron2.git@c470ca3
   omegaconf==2.1.1
   numpy
  ```
- `python .\install.py` (Must be Run as Administrator...)
- TODO: This is just WIP.