Repository: ggml-org/llama.cpp
Pull request: vulkan: handle device dedup on MacOS + Vega II Duo cards

Deduplication here relied on the fact that vulkan would return unique UUID for different physical GPUs. It is at the moment not always the case. On Mac Pro 2019 running Mac OS, with 2 Vega II Duo cards (so, 4 GPU total), MotlenVK would assign same UUID to pairs of GPUs, unless they are connected with Infinity Fabric.

See more details here: KhronosGroup/MoltenVK#2683.

The right way is to fix that in MoltenVK, but until it is fixed, llama.cpp would only recognize 2 of 4 GPUs in such configuration.

The deduplication logic here is changed to only filter GPUs if UUID is same but driver is different.

Before fix:
```
% rm -rf build && cmake -B build -DLLAMA_CURL=1 -DGGML_METAL=OFF -DGGML_VULKAN=1 -DCMAKE_BUILD_TYPE=Release
% cmake --build build --config Release -j 16
% ./build/bin/llama-cli --list-devices
ggml_vulkan: Found 2 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
Available devices:
  Vulkan0: AMD Radeon Pro Vega II Duo (32752 MiB, 32752 MiB free)
  Vulkan1: AMD Radeon Pro Vega II Duo (32752 MiB, 32752 MiB free)
  BLAS: Accelerate (0 MiB, 0 MiB free)
```

After fix:


```
% rm -rf build && cmake -B build -DLLAMA_CURL=1 -DGGML_METAL=OFF -DGGML_VULKAN=1 -DCMAKE_BUILD_TYPE=Release
% cmake --build build --config Release -j 16
% ./build/bin/llama-cli --list-devices
ggml_vulkan: Found 4 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 2 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 3 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
Available devices:
  Vulkan0: AMD Radeon Pro Vega II Duo (32752 MiB, 32752 MiB free)
  Vulkan1: AMD Radeon Pro Vega II Duo (32752 MiB, 32752 MiB free)
  Vulkan2: AMD Radeon Pro Vega II Duo (32752 MiB, 32752 MiB free)
  Vulkan3: AMD Radeon Pro Vega II Duo (32752 MiB, 32752 MiB free)
  BLAS: Accelerate (0 MiB, 0 MiB free)

```

Test generating with splitting across 4 GPUs:
```
(base) @tomb ~/projects/forks/llama.cpp % ./build/bin/llama-cli -m ~/projects/llms/Devstral-Small-2-24B-Instruct-2512-UD-Q8_K_XL.gguf -c 1024 -ngl 999 -sm layer -ts 1,1,1,1 -st -p "How are you?"
ggml_vulkan: Found 4 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 1 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 2 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none
ggml_vulkan: 3 = AMD Radeon Pro Vega II Duo (MoltenVK) | uma: 0 | fp16: 1 | bf16: 0 | warp size: 64 | shared memory: 65536 | int dot: 0 | matrix cores: none

...

[ Prompt: 15.4 t/s | Generation: 11.1 t/s ]

Exiting...
llama_memory_breakdown_print: | memory breakdown [MiB]        | total    free    self   model   context   compute       unaccounted |
llama_memory_breakdown_print: |   - Vulkan0 (Pro Vega II Duo) | 32752 = 26412 + (7941 =  7666 +      44 +     231) + 17592186042813 |
llama_memory_breakdown_print: |   - Vulkan1 (Pro Vega II Duo) | 32752 = 28569 + (5947 =  5678 +      40 +     229) + 17592186042651 |
llama_memory_breakdown_print: |   - Vulkan2 (Pro Vega II Duo) | 32752 = 28569 + (5947 =  5678 +      40 +     229) + 17592186042651 |
llama_memory_breakdown_print: |   - Vulkan3 (Pro Vega II Duo) | 32752 = 25766 + (7639 =  7337 +      36 +     266) + 17592186043761 |
llama_memory_breakdown_print: |   - Host                      |                  1292 =  1280 +       0 +      12                   |
```


Changed files:
filename: ggml/src/ggml-vulkan/ggml-vulkan.cpp
patch: @@ -5522,22 +5522,32 @@ static void ggml_vk_instance_init() {
 
             if ((new_props.properties.deviceType == vk::PhysicalDeviceType::eDiscreteGpu || new_props.properties.deviceType == vk::PhysicalDeviceType::eIntegratedGpu) && ggml_vk_device_is_supported(devices[i])) {
                 // Check if there are two physical devices corresponding to the same GPU
+                // This handles the case where the same GPU appears with different drivers (e.g., RADV + AMDVLK on Linux),
+                // see https://github.com/ggml-org/llama.cpp/pull/7582 for original deduplication.
+                // However, for MoltenVK on macOS, multiple GPUs on the same card may report the same UUID,
+                // see https://github.com/KhronosGroup/MoltenVK/issues/2683. Until this is fixed, we'll only deduplicate
+                // when drivers differ (same driver + same UUID = likely different GPUs)
                 auto old_device = std::find_if(
                     vk_instance.device_indices.begin(),
                     vk_instance.device_indices.end(),
-                    [&devices, &new_id](const size_t k){
+                    [&devices, &new_id, &new_driver](const size_t k){
                         vk::PhysicalDeviceProperties2 old_props;
+                        vk::PhysicalDeviceDriverProperties old_driver;
                         vk::PhysicalDeviceIDProperties old_id;
-                        old_props.pNext = &old_id;
+                        old_props.pNext = &old_driver;
+                        old_driver.pNext = &old_id;
                         devices[k].getProperties2(&old_props);
 
-                        bool equals = std::equal(std::begin(old_id.deviceUUID), std::end(old_id.deviceUUID), std::begin(new_id.deviceUUID));
-                        equals = equals || (
+                        bool same_uuid = std::equal(std::begin(old_id.deviceUUID), std::end(old_id.deviceUUID), std::begin(new_id.deviceUUID));
+                        same_uuid = same_uuid || (
                             old_id.deviceLUIDValid && new_id.deviceLUIDValid &&
                             std::equal(std::begin(old_id.deviceLUID), std::end(old_id.deviceLUID), std::begin(new_id.deviceLUID))
                         );
 
-                        return equals;
+                        // Only deduplicate if same UUID AND different drivers
+                        // (same driver + same UUID on MoltenVK = likely different GPUs on multi-GPU card)
+                        bool different_driver = (old_driver.driverID != new_driver.driverID);
+                        return same_uuid && different_driver;
                     }
                 );
                 if (old_device == vk_instance.device_indices.end()) {