Repository: pytorch
Issue: GPU Vendor-Agnosticism via Vulkan
Labels: feature,module: cuda,triaged,module: vulkan

## Feature

PyTorch shipping with ready-to-go GPU-vendor-agnostic functionality via the Vulkan API. The ability to run on Nvidia, AMD, and in the future Intel GPUs without hassle and with good performance.

## Motivation

PyTorch (and HPGPU compute in general) is captive to CUDA, and at the moment for good reason. Nvidia spends huge amounts of money and manpower on ensuring CUDA is well-documented and well-supported when implemented as the backend for numerous compute libraries and frameworks. However, this leaves the compute industry vulnerable to Nvidia's mistakes or, worst-case, bankruptcy (an extreme stretch, but a non-zero possibility regardless). Currently, their Ampere GPUs consume large amounts of power and feature downright confusing RAM pool amounts given their price points. Whether an Ampere Titan is coming remains to be seen.

AMD has announced and will within days of this post release seemingly very solid cards with large RAM pools and high performance at reasonable price points. Yet, their own ROCm compute libraries don't even support the previous generation of their RDNA cards, with AMD claiming the architecture is targeted at gaming. Still, with their performance and memory pools, they would make sense for many people as workstation compute cards.

Intel has GPUs in development, and are targeting both gaming and compute. Whether they will make any attempt at compatibility with CUDA or ROCm is an open question.

## Pitch

In order for a vendor to sell a viable GPU in 2020, their drivers must robustly support Vulkan. Now that Vulkan is adopting OpenCL's compute role and becoming a hybrid API, it is a very stable and well-supported target for ensuring code can run on any number of GPU architectures. Nvidia and AMD both support it first-class, and Intel has already demonstrated their GPUs running at the very least with Vulkan games; their HPGPU performance remains to be seen.

There is an active project known as Vulkan Kompute which aims to turn Vulkan's compute functionality and SPIR-V compilation into easily accessible Python and C++ APIs. This looks ready-made to be a fast and possibly even final target API for frameworks such as PyTorch and TensorFlow, not to mention other languages. If not for maximally-optimized deployment or commercial cloud training, then at least for workstations and getting up, running, and productive quickly.

https://kompute.cc/
https://github.com/EthicalML/vulkan-kompute/
https://towardsdatascience.com/beyond-cuda-gpu-accelerated-python-for-machine-learning-in-cross-vendor-graphics-cards-made-simple-6cc828a45cc3

Further optimizations can likely be made on a per-architecture basis using pure Vulkan C++ APIs.

## Alternatives

OpenCL is aging and largely losing support from high-performance GPU vendors.

ROCm is AMD's open-source implementation of CUDA which can be adapted to ostensibly any GPU architecture. However, AMD's own development is slow and they do not robustly support their newer Navi GPUs.

I do not know enough about OneAPI to comment except that I have seen some ambivalence about its performance results.

cc @ngimel @jeffdaily @sunway513 @jithunnair-amd @ROCmSupport @KyleCZH