Repository: ollama
Issue: Add support for amd Radeon 780M gfx1103 - override works
Labels: feature request,linux,windows,amd

### What are you trying to do?

Please support GPU acceleration using "AMD Ryzen 7 PRO 7840U w/ Radeon 780M Graphics" on Linux (Ubuntu 22.04).

Newer notebooks are shipped with AMD 7840U and support setting VRAM from 1GB to 8GB in the bios. With GPU acceleration only 1 vCPU is used and user experience with 7B models is quite good.

Not working ("amdgpu [0] gfx1103 is not supported"):

```
OLLAMA_LLM_LIBRARY="rocm_v60000" /usr/bin/ollama serve
time=2024-03-17T03:02:49.566+01:00 level=INFO source=images.go:806 msg="total blobs: 20"
time=2024-03-17T03:02:49.566+01:00 level=INFO source=images.go:813 msg="total unused blobs removed: 0"
time=2024-03-17T03:02:49.566+01:00 level=INFO source=routes.go:1110 msg="Listening on 127.0.0.1:11434 (version 0.1.29)"
time=2024-03-17T03:02:49.566+01:00 level=INFO source=payload_common.go:112 msg="Extracting dynamic libraries to /tmp/ollama3117537729/runners ..."
time=2024-03-17T03:02:51.423+01:00 level=INFO source=payload_common.go:139 msg="Dynamic LLM libraries [rocm_v60000 cpu_avx cpu_avx2 cuda_v11 cpu]"
time=2024-03-17T03:02:51.424+01:00 level=INFO source=gpu.go:77 msg="Detecting GPU type"
time=2024-03-17T03:02:51.424+01:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-03-17T03:02:51.427+01:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []"
time=2024-03-17T03:02:51.427+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-17T03:02:51.427+01:00 level=INFO source=amd_linux.go:50 msg="AMD Driver: 6.3.6"
time=2024-03-17T03:02:51.427+01:00 level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1103]"
time=2024-03-17T03:02:51.430+01:00 level=WARN source=amd_linux.go:114 msg="amdgpu [0] gfx1103 is not supported by /tmp/ollama3117537729/rocm [gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942]"
time=2024-03-17T03:02:51.430+01:00 level=WARN source=amd_linux.go:116 msg="See https://github.com/ollama/ollama/blob/main/docs/troubleshooting.md for HSA_OVERRIDE_GFX_VERSION usage"
time=2024-03-17T03:02:51.430+01:00 level=INFO source=amd_linux.go:127 msg="all detected amdgpus are skipped, falling back to CPU"
time=2024-03-17T03:02:51.430+01:00 level=INFO source=routes.go:1133 msg="no GPU detected"
```

My current workaround is to force gfx1102 (no issues so far):

```
HSA_OVERRIDE_GFX_VERSION="11.0.2" OLLAMA_LLM_LIBRARY="rocm_v60000" /usr/bin/ollama serve
time=2024-03-17T03:04:54.436+01:00 level=INFO source=images.go:806 msg="total blobs: 20"
time=2024-03-17T03:04:54.436+01:00 level=INFO source=images.go:813 msg="total unused blobs removed: 0"
time=2024-03-17T03:04:54.437+01:00 level=INFO source=routes.go:1110 msg="Listening on 127.0.0.1:11434 (version 0.1.29)"
time=2024-03-17T03:04:54.437+01:00 level=INFO source=payload_common.go:112 msg="Extracting dynamic libraries to /tmp/ollama2902102366/runners ..."
time=2024-03-17T03:04:56.315+01:00 level=INFO source=payload_common.go:139 msg="Dynamic LLM libraries [cpu_avx2 cuda_v11 rocm_v60000 cpu cpu_avx]"
time=2024-03-17T03:04:56.315+01:00 level=INFO source=gpu.go:77 msg="Detecting GPU type"
time=2024-03-17T03:04:56.315+01:00 level=INFO source=gpu.go:191 msg="Searching for GPU management library libnvidia-ml.so"
time=2024-03-17T03:04:56.317+01:00 level=INFO source=gpu.go:237 msg="Discovered GPU libraries: []"
time=2024-03-17T03:04:56.317+01:00 level=INFO source=cpu_common.go:11 msg="CPU has AVX2"
time=2024-03-17T03:04:56.317+01:00 level=INFO source=amd_linux.go:50 msg="AMD Driver: 6.3.6"
time=2024-03-17T03:04:56.318+01:00 level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1103]"
time=2024-03-17T03:04:56.319+01:00 level=INFO source=amd_linux.go:246 msg="[0] amdgpu totalMemory 8192M"
time=2024-03-17T03:04:56.319+01:00 level=INFO source=amd_linux.go:247 msg="[0] amdgpu freeMemory  8192M"
```

Note: using "rocm_v6" was not working for me, so I chose "rocm_v60000" (ls /tmp/ollama758986631/runners/ gives
cpu  cpu_avx  cpu_avx2  cuda_v11  rocm_v60000)

Some benchmarks using 7840U (numbers from second run), ubuntu 22.04, kernel 6.5, vram switched to 8GB in bios:

```
> CPU
OLLAMA_LLM_LIBRARY="cpu_avx2" /usr/bin/ollama serve
ollama run llama2:latest "where was beethoven born?" --verbose
Ludwig van Beethoven was born in Bonn, Germany on December 16, 1770.
total duration:       4.343514826s
load duration:        264.691µs
prompt eval duration: 168.205ms
prompt eval rate:     0.00 tokens/s
eval count:           26 token(s)
eval duration:        4.174563s
eval rate:            6.23 tokens/s
```

``` 
> GPU
HSA_OVERRIDE_GFX_VERSION="11.0.2" OLLAMA_LLM_LIBRARY="rocm_v60000u_avx2" /usr/bin/ollama serve
ollama run llama2:latest "where was beethoven born?" --verbose
Ludwig van Beethoven was born in Bonn, Germany on December 16, 1770.
total duration:       1.513455927s
load duration:        161.535µs
prompt eval duration: 65.979ms
prompt eval rate:     0.00 tokens/s
eval count:           27 token(s)
eval duration:        1.446805s
eval rate:            18.66 tokens/s
```

### How should we solve this?

_No response_

### What is the impact of not solving this?

_No response_

### Anything else?

_No response_