PyTorch 2.0 not working on Windows
Labels: module: windows, triaged, oncall: pt2

### 🐛 Describe the bug

When I try installing the nightly build with the following command:

`pip3 install numpy --pre torch[dynamo] --force-reinstall --extra-index-url https://download.pytorch.org/whl/nightly/cu117`

It gives the following warning

`WARNING: torch 2.0.0.dev20221213+cu117 does not provide the extra 'dynamo'`

And when I try to check the installation with 

```
git clone https://github.com/pytorch/pytorch
cd tools/dynamo
python verify_dynamo.py
```

This results in the following error

```
Traceback (most recent call last):
  File ".\verify_dynamo.py", line 167, in <module>
    main()
  File ".\verify_dynamo.py", line 155, in main
    cuda_ver = check_cuda()
  File ".\verify_dynamo.py", line 64, in check_cuda
    cuda_ver = get_cuda_version()
  File ".\verify_dynamo.py", line 39, in get_cuda_version
    raise VerifyDynamoError(cpp_extension.CUDA_NOT_FOUND_MESSAGE)
__main__.VerifyDynamoError:
CUDA was not found on the system, please set the CUDA_HOME or the CUDA_PATH
environment variable or add NVCC to your system PATH. The extension compilation will fail.
```

The torch installation is working but dynamo seems not to work. When I run the benchmarks from [https://gist.github.com/Chillee/f86675147366a7a0c6e244eaa78660f7](https://gist.github.com/Chillee/f86675147366a7a0c6e244eaa78660f7) I get the following 

```
C:\Users\user\anaconda3\envs\auxiliar\lib\site-packages\torch\_dynamo\eval_frame.py:428: UserWarning: Windows is not currently supported, torch._dynamo.optimize() will do nothing
  warnings.warn(
eager: 1386.2895965576172us
PT 2.0: 1406.266689300537us
```

Which again shows that dynamo is not working. If I try installing torchdynamo apart, it gives this error

```
Installing collected packages: torchdynamo
  Running setup.py install for torchdynamo ... error
  error: subprocess-exited-with-error

  × Running setup.py install for torchdynamo did not run successfully.
  │ exit code: 1
  ╰─> [108 lines of output]
      running install
      C:\Users\user\anaconda3\envs\auxiliar\lib\site-packages\setuptools\command\install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-38
      creating build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\allowed_functions.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\bytecode_analysis.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\bytecode_transformation.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\codegen.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\config.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\convert_frame.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\debug_utils.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\eval_frame.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\exc.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\guards.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\logging.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\mutation_guard.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\output_graph.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\profiler.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\replay_record.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\resume_execution.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\side_effects.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\skipfiles.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\source.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\symbolic_convert.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\testing.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\utils.py -> build\lib.win-amd64-cpython-38\torchdynamo
      copying torchdynamo\__init__.py -> build\lib.win-amd64-cpython-38\torchdynamo
      creating build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\codecache.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\compile_fx.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\config.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\debug.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\decomposition.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\dependencies.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\exc.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\graph.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\ir.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\lowering.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\metrics.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\overrides.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\scheduler.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\sizevars.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\utils.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\virtualized.py -> build\lib.win-amd64-cpython-38\torchinductor
      copying torchinductor\__init__.py -> build\lib.win-amd64-cpython-38\torchinductor
      creating build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\analysis.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\backends.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\distributed.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\inference.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\log_args.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\normalize.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\subgraph.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\training.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      copying torchdynamo\optimizations\__init__.py -> build\lib.win-amd64-cpython-38\torchdynamo\optimizations
      creating build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\base.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\builder.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\builtin.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\constant.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\dicts.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\functions.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\lists.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\misc.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\nn_module.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\tensor.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\torch.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\user_defined.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      copying torchdynamo\variables\__init__.py -> build\lib.win-amd64-cpython-38\torchdynamo\variables
      creating build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\autotuner.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\common.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\cpp.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\triton.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\triton_template.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\wrapper.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\__init__.py -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      creating build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\autotune.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\batched_matmul.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\conv.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\conv1x1.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\conv_perf_model.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\matmul.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\mm_perf_model.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\utils.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\triton_ops\__init__.py -> build\lib.win-amd64-cpython-38\torchinductor\triton_ops
      copying torchinductor\codegen\cpp_prefix.h -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\triton_conv_delta_x.j2 -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\triton_conv_delta_x_hwc.j2 -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      copying torchinductor\codegen\triton_mm.j2 -> build\lib.win-amd64-cpython-38\torchinductor\codegen
      running build_ext
      building 'torchdynamo._eval_frame' extension
      creating build\temp.win-amd64-cpython-38
      creating build\temp.win-amd64-cpython-38\Release
      creating build\temp.win-amd64-cpython-38\Release\torchdynamo
      "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\user\anaconda3\envs\auxiliar\include -IC:\Users\user\anaconda3\envs\auxiliar\Include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /Tctorchdynamo/_eval_frame.c /Fobuild\temp.win-amd64-cpython-38\Release\torchdynamo/_eval_frame.obj -Wall
      _eval_frame.c
      C:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt\corecrt_io.h(49): warning C4820: '_finddata32i64_t': '4' bytes de relleno agregados despu‚s de miembro de datos 'name'
      C:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt\corecrt_io.h(54): warning C4820: '_finddata64i32_t': '4' bytes de relleno agregados despu‚s de miembro de datos 'attrib'
      C:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt\corecrt_io.h(64): warning C4820: '__finddata64_t': '4' bytes de relleno agregados despu‚s de miembro de datos 'attrib'
      C:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt\corecrt_io.h(69): warning C4820: '__finddata64_t': '4' bytes de relleno agregados despu‚s de miembro de datos 'name'
      c:\users\user\anaconda3\envs\auxiliar\include\pyconfig.h(205): fatal error C1083: No se puede abrir el archivo incluir: 'basetsd.h': No such file or directory
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit code 2
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> torchdynamo

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
```

I am using Windows 10, with AMD64 processors (the wheels for python show win_amd64 at the end). I also use 2 NVIDIA RTX 3090 with CUDA 12.0. 

### Versions

Collecting environment information...
PyTorch version: 2.0.0.dev20221213+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 10 Pro
GCC version: Could not collect
Clang version: Could not collect
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.15 (default, Nov 24 2022, 14:38:14) [MSC v.1916 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-10-10.0.19045-SP0
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 526.86
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.24.0rc2
[pip3] torch==2.0.0.dev20221213+cu117
[conda] numpy                     1.24.0rc2                pypi_0    pypi
[conda] torch                     2.0.0.dev20221213+cu117          pypi_0    pypi

cc @peterjc123 @mszhanyi @skyline75489 @nbcsm @ezyang @soumith @msaroufim @wconstab @ngimel @bdhirsh