Repository: cupy/cupy

Issue:
Automatically infer HCC_AMDGPU_TARGET
- build: Support `CUPY_NVCC_GENERATE_CODE=current` equivalent in ROCm to set `HCC_AMDGPU_TARGET` automatically
- runtime: Automatically infer `HCC_AMDGPU_TARGET` (this environment variable is only needed for hipcc case?)


Discussion:
> * runtime: Automatically infer `HCC_AMDGPU_TARGET` (this environment variable is only needed for hipcc case?)

I doubt we need it at runtime. There was a time I forgot to set it with a fresh local build, but CuPy still ran fine:
https://github.com/cupy/cupy/blob/6a663512ee888c49ef6542fd065a6046fa9e9353/cupy/cuda/compiler.py#L761-L767
It should be readily testable by running `test_raw.py` without the env var set. Both `hiprtc` and `hipcc` got tested.
> * build: Support `CUPY_NVCC_GENERATE_CODE=current` equivalent in ROCm to set `HCC_AMDGPU_TARGET` automatically

I just did a quick test locally. It seems if only building for local use, as mentioned in the above linked comment `hipcc` can also infer `HCC_AMDGPU_TARGET` correctly. I confirmed this by removing the `cub` and `thrust` modules, unset both `HCC_AMDGPU_TARGET` and `__HIP_PLATFORM_HCC__`, and rebuild. It builds just fine.

After that, we can examine the binary by ROCm's built-in tool (similar to `cuobjdump` on CUDA):
```
$ extractkernel -i cupy/cuda/thrust.cpython-37m-x86_64-linux-gnu.so
```
which generates three files, two of which are for my arch (gfx906):
```
$ cd cupy/cuda; ls -lh thrust.cp*
-rw-rw-r-- 1 leofang leofang 328K Feb 23 16:16 thrust.cpp
-rwxrwxr-x 1 leofang leofang 7.6M Feb 23 16:19 thrust.cpython-37m-x86_64-linux-gnu.so
-rw-rw-r-- 1 leofang leofang 6.2M Feb 23 16:20 thrust.cpython-37m-x86_64-linux-gnu.so-000.bundle
-rw-rw-r-- 1 leofang leofang 6.2M Feb 23 16:20 thrust.cpython-37m-x86_64-linux-gnu.so-000-gfx906.hsaco
-rw-rw-r-- 1 leofang leofang  73M Feb 23 16:20 thrust.cpython-37m-x86_64-linux-gnu.so-000-gfx906.isa
```

Could be relevant to #4720.
>> > * runtime: Automatically infer `HCC_AMDGPU_TARGET` (this environment variable is only needed for hipcc case?)
>> 
>> I doubt we need it at runtime. There was a time I forgot to set it with a fresh local build, but CuPy still ran fine:
>> 
>> https://github.com/cupy/cupy/blob/6a663512ee888c49ef6542fd065a6046fa9e9353/cupy/cuda/compiler.py#L761-L767
>
> It should be readily testable by running `test_raw.py` without the env var set. Both `hiprtc` and `hipcc` got tested.

OK I tested this locally without setting `HCC_AMDGPU_TARGET` and `__HIP_PLATFORM_HCC__`. By removing the checks for `HCC_AMDGPU_TARGET` in `cupy/cuda/compiler.py` all checks passed on ROCm 3.5.0. Let me know if you think it's ok to remove them and I will send a PR.
I skimmed hipcc code (it's written in perl 😅 ), it seems the output from `/opt/rocm/bin/rocm_agent_enumerator -t GPU` (which prints arch for each GPU installed) is used when `HCC_AMDGPU_TARGET` is unset. I'm curious how hiprtc would behave, though.
> I skimmed hipcc code (it's written in perl 😅 ), it seems the output from `/opt/rocm/bin/rocm_agent_enumerator -t GPU` (which prints arch for each GPU installed) is used when `HCC_AMDGPU_TARGET` is unset. 

Cool, thanks @kmaehashi. Pointer: https://github.com/ROCm-Developer-Tools/HIP/blob/2080cc113a2d767352b512b9d24c0620b6dee790/bin/hipcc#L717-L728

> I'm curious how hiprtc would behave, though.

hipRTC first queries the current device, and then delegates the actual compilation to the ROCclr backend:
https://github.com/ROCm-Developer-Tools/HIP/blob/2080cc113a2d767352b512b9d24c0620b6dee790/rocclr/hip_rtc.cpp#L225
https://github.com/ROCm-Developer-Tools/ROCclr/blob/5cefcaf62893fcd86c8feed6bb1ebb84850fcd2f/platform/program.cpp#L477

So looks like we're good to remove the advice of setting `HCC_AMDGPU_TARGET` at runtime.

Resolving patch:
diff --git a/cupy/cuda/compiler.py b/cupy/cuda/compiler.py
--- a/cupy/cuda/compiler.py
+++ b/cupy/cuda/compiler.py
@@ -234,13 +234,11 @@ def _jitify_prep(source, options, cu_path):
 def compile_using_nvrtc(source, options=(), arch=None, filename='kern.cu',
                         name_expressions=None, log_stream=None,
                         cache_in_memory=False, jitify=False):
-    if not arch:
-        arch = _get_arch()
-
+    # For hipRTC, arch is ignored
     if not runtime.is_hip:
+        if not arch:
+            arch = _get_arch()
         options += ('-arch=compute_{}'.format(arch),)
-    else:
-        options += ('-arch={}'.format(arch),)
 
     def _compile(
             source, options, cu_path, name_expressions, log_stream, jitify):
@@ -643,12 +641,10 @@ def is_valid_kernel_name(name):
 
 
 def compile_using_hipcc(source, options, arch, log_stream=None):
-    # TODO(leofang): it seems as of ROCm 3.5.0 hiprtc/hipcc can automatically
-    # pick up the right arch without needing HCC_AMDGPU_TARGET. Perhaps we just
-    # don't bother passing arch to hiprtc/hipcc?
-    assert len(arch) > 0
-    # pass HCC_AMDGPU_TARGET same as arch
-    cmd = ['hipcc', '--genco', '-arch='+arch] + list(options)
+    # As of ROCm 3.5.0 hiprtc/hipcc can automatically pick up the
+    # right arch without setting HCC_AMDGPU_TARGET, so we don't need
+    # to set arch here
+    cmd = ['hipcc', '--genco'] + list(options)
 
     with tempfile.TemporaryDirectory() as root_dir:
         path = os.path.join(root_dir, 'kern')
@@ -758,13 +754,14 @@ def _compile_with_cache_hip(source, options, arch, cache_dir, extra_source,
 
     if cache_dir is None:
         cache_dir = get_cache_dir()
-    # TODO(leofang): it seems as of ROCm 3.5.0 hiprtc/hipcc can automatically
-    # pick up the right arch without needing HCC_AMDGPU_TARGET. Perhaps we just
-    # don't bother passing arch to hiprtc/hipcc?
+    # As of ROCm 3.5.0 hiprtc/hipcc can automatically pick up the
+    # right arch without setting HCC_AMDGPU_TARGET, so we don't need
+    # to tell the compiler which arch we are targeting. But, we still
+    # need to know arch as part of the cache key:
     if arch is None:
-        arch = os.environ.get('HCC_AMDGPU_TARGET')
-        if arch is None:
-            raise RuntimeError('HCC_AMDGPU_TARGET is not set')
+        # On HIP, gcnArch is computed from "compute capability":
+        # https://github.com/ROCm-Developer-Tools/HIP/blob/rocm-4.0.0/rocclr/hip_device.cpp#L202
+        arch = device.Device().compute_capability
     if use_converter:
         source = _convert_to_hip_source(source, extra_source,
                                         is_hiprtc=(backend == 'hiprtc'))


Test patch: