Repository: pytorch
Issue: Building PyTorch with Vulkan backend fails (1.13 and master)
Labels: triaged,module: vulkan

### 🐛 Describe the bug

Trying to build pytorch on linux with vulkan as the backend following these instructions (https://pytorch.org/tutorials/prototype/vulkan_workflow.html) fails. Just for reference, what I ran to try building:
```
export USE_VULKAN=1
export USE_VULKAN_SHADERC_RUNTIME=1
export USE_VULKAN_WRAPPER=0
export USE_CUDA=0
export USE_ROCM=0
python3 setup.py bdist_wheel
```

For 1.13
The first error encountered is already described here: https://github.com/pytorch/pytorch/issues/89105
If the `layernorm.glsl` file is updated to remove the quotation marks referred to in the linked issue, I eventually hit the following error:

```
[5403/6393] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/codegen/onednn/graph_helper.cpp.o
FAILED: caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/codegen/onednn/graph_helper.cpp.o 
/usr/bin/c++ -DAT_PER_OPERATOR_HEADERS -DBUILD_ONEDNN_GRAPH -DCPUINFO_SUPPORTED_PLATFORM=1 -DFMT_HEADER_ONLY=1 -DFXDIV_USE_INLINE_ASSEMBLY=0 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DMINIZ_DISABLE_ZIP_READER_CRC32_CHECKS -DNNP_CONVOLUTION_ONLY=0 -DNNP_INFERENCE_ONLY=0 -DONNXIFI_ENABLE_EXT=1 -DONNX_ML=1 -DONNX_NAMESPACE=onnx_torch -DUSE_C10D_GLOO -DUSE_DISTRIBUTED -DUSE_EXTERNAL_MZCRC -DUSE_RPC -DUSE_TENSORPIPE -D_FILE_OFFSET_BITS=64 -Dtorch_cpu_EXPORTS -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/aten/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/build -I/home/me/Dev/scratch/pytorch_vk/pytorch -I/home/me/Dev/scratch/pytorch_vk/pytorch/cmake/../third_party/benchmark/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/onnx -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/third_party/onnx -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/foxi -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/third_party/foxi -I/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/api -I/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/api/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/caffe2/aten/src/TH -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/caffe2/aten/src/TH -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/caffe2/aten/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/caffe2/../aten/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/miniz-2.1.0 -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/kineto/libkineto/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/kineto/libkineto/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/vulkan -I/home/me/Dev/scratch/pytorch_vk/pytorch/aten/../third_party/VulkanMemoryAllocator -I/home/me/Dev/scratch/pytorch_vk/pytorch/aten/../third_party/catch/single_include -I/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/.. -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/FXdiv/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/c10/.. -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/pthreadpool/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/cpuinfo/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/QNNPACK/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/quantized/cpu/qnnpack/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/quantized/cpu/qnnpack/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/cpuinfo/deps/clog/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/NNPACK/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/fbgemm/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/fbgemm -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/fbgemm/third_party/asmjit/src -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/ittapi/src/ittnotify -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/FP16/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/tensorpipe -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/third_party/tensorpipe -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/tensorpipe/third_party/libnop/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/fmt/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/build/third_party/ideep/mkl-dnn/third_party/oneDNN/include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/ideep/mkl-dnn/third_party/oneDNN/src/../include -I/home/me/Dev/scratch/pytorch_vk/pytorch/third_party/flatbuffers/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/build/third_party/gloo -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/cmake/../third_party/gloo -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/cmake/../third_party/googletest/googlemock/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/cmake/../third_party/googletest/googletest/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/protobuf/src -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/gemmlowp -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/neon2sse -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/XNNPACK/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/ittapi/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/cmake/../third_party/eigen -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/ideep/mkl-dnn/third_party/oneDNN/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/ideep/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/third_party/ideep/mkl-dnn/include -isystem /home/me/Dev/scratch/pytorch_vk/pytorch/build/include -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DUSE_VULKAN -DUSE_VULKAN_API -DUSE_VULKAN_SHADERC_RUNTIME -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow -DHAVE_AVX2_CPU_DEFINITION -O3 -DNDEBUG -DNDEBUG -fPIC -DCAFFE2_USE_GLOO -DTH_HAVE_THREAD -Wall -Wextra -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-missing-field-initializers -Wno-write-strings -Wno-unknown-pragmas -Wno-type-limits -Wno-array-bounds -Wno-sign-compare -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-missing-braces -Wno-maybe-uninitialized -fvisibility=hidden -O2 -fopenmp -DCAFFE2_BUILD_MAIN_LIB -pthread -DASMJIT_STATIC -std=gnu++14 -MD -MT caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/codegen/onednn/graph_helper.cpp.o -MF caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/codegen/onednn/graph_helper.cpp.o.d -o caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/codegen/onednn/graph_helper.cpp.o -c /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/graph_helper.cpp
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/graph_helper.cpp: In function ‘torch::jit::fuser::onednn::Operator torch::jit::fuser::onednn::createOperator(torch::jit::Node*)’:
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/graph_helper.cpp:168:42: error: ‘HardTanh’ is not a member of ‘torch::jit::fuser::onednn::opkind’ {aka ‘dnnl::graph::op::kind’}
  168 |       return makeEltwiseOp(node, opkind::HardTanh)
      |                                          ^~~~~~~~
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/graph_helper.cpp:173:42: error: ‘HardTanh’ is not a member of ‘torch::jit::fuser::onednn::opkind’ {aka ‘dnnl::graph::op::kind’}
  173 |       return makeEltwiseOp(node, opkind::HardTanh)
      |                                          ^~~~~~~~
[5408/6393] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/jit/codegen/onednn/interface.cpp.o
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/interface.cpp: In function ‘torch::jit::Operation torch::jit::createLlgaKernel(const torch::jit::Node*)’:
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/interface.cpp:97:3: warning: ‘torch::jit::Operation::Operation(F&&) [with F = torch::jit::createLlgaKernel(const torch::jit::Node*)::<lambda(torch::jit::Stack*)>; typename std::enable_if<std::is_constructible<std::function<void(std::vector<c10::IValue>*)>, F&&>::value, int>::type <anonymous> = 0]’ is deprecated: Please use void(Stack&) to register operator instead. [-Wdeprecated-declarations]
   97 |   };
      |   ^
In file included from /home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/core/boxing/KernelFunction.h:5,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/core/dispatch/Dispatcher.h:4,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/runtime/operator.h:6,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/ir/ir.h:7,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/defer_size_check.h:3,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/interface.cpp:2:
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/core/stack.h:25:3: note: declared here
   25 |   Operation(F&& raw): op_([raw = std::forward<F>(raw)](Stack& stack) {
      |   ^~~~~~~~~
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/interface.cpp: In function ‘torch::jit::Operation torch::jit::createLlgaGuardKernel(const torch::jit::Node*)’:
/home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/interface.cpp:162:3: warning: ‘torch::jit::Operation::Operation(F&&) [with F = torch::jit::createLlgaGuardKernel(const torch::jit::Node*)::<lambda(torch::jit::Stack*)>; typename std::enable_if<std::is_constructible<std::function<void(std::vector<c10::IValue>*)>, F&&>::value, int>::type <anonymous> = 0]’ is deprecated: Please use void(Stack&) to register operator instead. [-Wdeprecated-declarations]
  162 |   };
      |   ^
In file included from /home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/core/boxing/KernelFunction.h:5,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/core/dispatch/Dispatcher.h:4,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/runtime/operator.h:6,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/ir/ir.h:7,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/defer_size_check.h:3,
                 from /home/me/Dev/scratch/pytorch_vk/pytorch/torch/csrc/jit/codegen/onednn/interface.cpp:2:
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/core/stack.h:25:3: note: declared here
   25 |   Operation(F&& raw): op_([raw = std::forward<F>(raw)](Stack& stack) {
      |   ^~~~~~~~~
[5415/6393] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/torch/csrc/autograd/generated/VariableType_2.cpp.o
ninja: build stopped: subcommand failed.
```

I also tried the master branch since there seems be a bunch of changes in the vulkan files. I hit the following error when trying to bulid off master:
```
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp: In function ‘at::native::vulkan::api::ShaderSource at::native::vulkan::ops::conv2d::get_shader(at::IntArrayRef, at::IntArrayRef, at::IntArrayRef, at::IntArrayRef, at::native::vulkan::ops::Conv2dMethod, bool, bool)’:
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:299:28: error: ‘quantized_conv2d’ was not declared in this scope; did you mean ‘quantized_conv2d_glsl’?
  299 |         shader = VK_SHADER(quantized_conv2d);
      |                            ^~~~~~~~~~~~~~~~
      |                            quantized_conv2d_glsl
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:299:18: error: ‘VK_SHADER’ was not declared in this scope
  299 |         shader = VK_SHADER(quantized_conv2d);
      |                  ^~~~~~~~~
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:302:28: error: ‘quantized_conv2d_dw’ was not declared in this scope; did you mean ‘quantized_conv2d_glsl’?
  302 |         shader = VK_SHADER(quantized_conv2d_dw);
      |                            ^~~~~~~~~~~~~~~~~~~
      |                            quantized_conv2d_glsl
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:305:28: error: ‘quantized_conv2d_pw_2x2’ was not declared in this scope; did you mean ‘quantized_conv2d_pw_2x2_glsl’?
  305 |         shader = VK_SHADER(quantized_conv2d_pw_2x2);
      |                            ^~~~~~~~~~~~~~~~~~~~~~~
      |                            quantized_conv2d_pw_2x2_glsl
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:313:24: error: ‘conv_transpose2d’ was not declared in this scope; did you mean ‘conv_transpose2d_glsl’?
  313 |     shader = VK_SHADER(conv_transpose2d);
      |                        ^~~~~~~~~~~~~~~~
      |                        conv_transpose2d_glsl
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:313:14: error: ‘VK_SHADER’ was not declared in this scope
  313 |     shader = VK_SHADER(conv_transpose2d);
      |              ^~~~~~~~~
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:319:32: error: expected primary-expression before ‘)’ token
  319 |       shader = VK_SHADER(conv2d);
      |                                ^
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:319:16: error: ‘VK_SHADER’ was not declared in this scope
  319 |       shader = VK_SHADER(conv2d);
      |                ^~~~~~~~~
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:322:26: error: ‘conv2d_dw’ was not declared in this scope; did you mean ‘qconv2d_vk’?
  322 |       shader = VK_SHADER(conv2d_dw);
      |                          ^~~~~~~~~
      |                          qconv2d_vk
/home/me/Dev/scratch/pytorch_vk/pytorch/aten/src/ATen/native/vulkan/ops/Convolution.cpp:325:26: error: ‘conv2d_pw_2x2’ was not declared in this scope
  325 |       shader = VK_SHADER(conv2d_pw_2x2);
      |                          ^~~~~~~~~~~~~
[5369/6406] Building CXX object caffe2/CMakeFiles/torch_cpu.dir/__/aten/src/ATen/native/cpu/BinaryOpsKernel.cpp.DEFAULT.cpp.o
ninja: build stopped: subcommand failed.
```

### Versions

Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.25.0
Libc version: glibc-2.35

Python version: 3.10.6 (main, Nov  2 2022, 18:53:38) [GCC 11.3.0] (64-bit runtime)
Python platform: Linux-5.15.0-53-generic-x86_64-with-glibc2.35
Is CUDA available: N/A
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: N/A

Versions of relevant libraries:
[pip3] numpy==1.23.5
[conda] Could not collect