Repository: ggml-org/llama.cpp
Pull request: context : use n_embd_out for pooled embedding extraction

## Summary

The MEAN/CLS/LAST pooling paths in `encode()` and `decode()` use `n_embd` (set to `hparams.n_embd_inp()`) to read from the pooled embedding tensor `t_embd`. For models with deepstack layers (qwen3vl), `n_embd_inp()` returns 16384 while the pooled tensor only has `n_embd_out()` = 4096 floats per sequence, causing:

GGML_ASSERT(offset + size <= ggml_nbytes(tensor) && "tensor read out of bounds")

The fix uses `hparams.n_embd_out()` instead, consistent with how `POOLING_TYPE_NONE` (line 1756) already handles it.

## Affected models

Qwen3-VL-Embedding-8B (and any model with `n_deepstack_layers > 0`)

## Test

Tested with `Qwen3-VL-Embedding-8B-Q4_K_M.gguf`, `--embedding --pooling last`:
- Before: crash on startup
- After: correct 4096-dim L2-normalized embeddings

No impact on models without deepstack (`n_embd_inp == n_embd_out`).

## Use case

Qwen3-VL-Embedding can create embeddings for both images and text in the same vector space. A typical workflow is to index images using the full model on GPU, then run text-only retrieval queries against that index using a quantized GGUF on CPU. This fix makes the second step possible.

## Disclosure

AI was used as a research/debugging aid to locate the root cause.


Changed files:
filename: src/llama-context.cpp
patch: @@ -1347,8 +1347,11 @@ int llama_context::encode(const llama_batch & batch_inp) {
                         const llama_seq_id seq_id  = ubatch.seq_id_unq[s];
                         const int32_t      seq_idx = ubatch.seq_idx[seq_id];
 
-                        embd_seq_out[seq_id].resize(n_embd);
-                        ggml_backend_tensor_get_async(backend_embd, t_embd, embd_seq_out[seq_id].data(), (n_embd*seq_idx)*sizeof(float), n_embd*sizeof(float));
+                        // use n_embd_out (not n_embd_inp) - the pooled embedding has the model's
+                        // output dimension, which differs from input dimension for deepstack models (e.g. qwen3vl)
+                        const uint32_t n_embd_out = hparams.n_embd_out();
+                        embd_seq_out[seq_id].resize(n_embd_out);
+                        ggml_backend_tensor_get_async(backend_embd, t_embd, embd_seq_out[seq_id].data(), (n_embd_out*seq_idx)*sizeof(float), n_embd_out*sizeof(float));
                     }
                 } break;
             case LLAMA_POOLING_TYPE_RANK:
@@ -1769,12 +1772,16 @@ int llama_context::decode(const llama_batch & batch_inp) {
                         // extract sequence embeddings (cleared before processing each batch)
                         auto & embd_seq_out = embd_seq;
 
+                        // use n_embd_out (not n_embd_inp) - the pooled embedding has the model's
+                        // output dimension, which differs from input dimension for deepstack models (e.g. qwen3vl)
+                        const uint32_t n_embd_out = hparams.n_embd_out();
+
                         for (uint32_t s = 0; s < ubatch.n_seqs_unq; ++s) {
                             const llama_seq_id seq_id  = ubatch.seq_id_unq[s];
                             const int32_t      seq_idx = ubatch.seq_idx[seq_id];
 
-                            embd_seq_out[seq_id].resize(n_embd);
-                            ggml_backend_tensor_get_async(backend_embd, t_embd, embd_seq_out[seq_id].data(), (n_embd*seq_idx)*sizeof(float), n_embd*sizeof(float));
+                            embd_seq_out[seq_id].resize(n_embd_out);
+                            ggml_backend_tensor_get_async(backend_embd, t_embd, embd_seq_out[seq_id].data(), (n_embd_out*seq_idx)*sizeof(float), n_embd_out*sizeof(float));
                         }
                     } break;
                 case LLAMA_POOLING_TYPE_RANK: