Repository: open-compass/VLMEvalKit
Issue: min_pixels of Qwen2.5-VL
Root cause: Documentation Deficiency

@mfarre  Hello, I'd like to know if there's any basis for setting `min_pixels=1280 * 28 * 28` for Qwen2.5-VL in this public eval tool? How can we determine if the results obtained using this value are comparable to results from other publications that use different eval methods?

Looking forward to your reply!

Thanks for reaching out!

I did work on Qwen VL evals but I don't remember authoring those on
VLMEvalKit.
Looking at code snippets everywhere (i.e. huggging face model cards)
min_pixels is usually set to a smaller number: 256 * 28 * 28.  I would say
that by setting it to 1280 * 28 * 28 one might get better results as we are
not giving the model tiny token budgets. That might be the rationale behind
that number but again I believe we did not take it from any official
reference.

Miquel



On Mon, Dec 22, 2025 at 1:21 AM Guanyu Zhou ***@***.***>
wrote:

> *The-Martyr* created an issue (open-compass/VLMEvalKit#1370)
> <https://github.com/open-compass/VLMEvalKit/issues/1370>
>
> @mfarre <https://github.com/mfarre> Hello, I'd like to know if there's
> any basis for setting min_pixels=1280 * 28 * 28 for Qwen2.5-VL in this
> public eval tool? How can we determine if the results obtained using this
> value are comparable to results from other publications that use different
> eval methods?
>
> Looking forward to your reply!
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/open-compass/VLMEvalKit/issues/1370>, or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAFAUYHJDLYPTJM3HAASAOL4C42Q3AVCNFSM6AAAAACPWOQJOOVHI2DSMVQWIX3LMV43ASLTON2WKOZTG42TCOBRGU4DGNA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


> Thanks for reaching out!
> 
> I did work on Qwen VL evals but I don't remember authoring those on
> VLMEvalKit.
> Looking at code snippets everywhere (i.e. huggging face model cards)
> min_pixels is usually set to a smaller number: 256 * 28 * 28.  I would say
> that by setting it to 1280 * 28 * 28 one might get better results as we are
> not giving the model tiny token budgets. That might be the rationale behind
> that number but again I believe we did not take it from any official
> reference.
> 
> Miquel
> […](#)

Could it be understood this way: 1280*28*28 is the upper bound recommended by Qwen for balancing performance, speed, and memory, while 16384 * 28 * 28 is its theoretically acceptable maximum value. 1280 resizes excessively small images, resulting in better performance for tasks requiring high resolution, such as document understanding. 

I tested this on some benchmarks and found that setting 1280 and 16384 yielded slightly better results than setting 256 and 1280. This aligns with intuition. 

However, considering fairness with other publishers, do you think it's better to maintain 1280 and 16384, or to use 256 and 1280?

thanks, maybe you can send a PR editing the readme explaining that the 1280
might favour better results than the 256 recommended in model cards /
tutorials? at least it is more visible.

On Mon, Dec 22, 2025 at 1:49 PM Guanyu Zhou ***@***.***>
wrote:

> *The-Martyr* left a comment (open-compass/VLMEvalKit#1370)
> <https://github.com/open-compass/VLMEvalKit/issues/1370#issuecomment-3681973980>
>
> Thanks for reaching out!
>
> I did work on Qwen VL evals but I don't remember authoring those on
> VLMEvalKit.
> Looking at code snippets everywhere (i.e. huggging face model cards)
> min_pixels is usually set to a smaller number: 256 * 28 * 28. I would say
> that by setting it to 1280 * 28 * 28 one might get better results as we are
> not giving the model tiny token budgets. That might be the rationale behind
> that number but again I believe we did not take it from any official
> reference.
>
> Miquel
> … <#m_-2793896884718794001_>
>
> Could it be understood this way: 1280*28*28 is the upper bound
> recommended by Qwen for balancing performance, speed, and memory, while
> 16384 * 28 * 28 is its theoretically acceptable maximum value. 1280 resizes
> excessively small images, resulting in better performance for tasks
> requiring high resolution, such as document understanding.
>
> I tested this on some benchmarks and found that setting 1280 and 16384
> yielded slightly better results than setting 256 and 1280. This aligns with
> intuition.
>
> However, considering fairness with other publishers, do you think it's
> better to maintain 1280 and 16384, or to use 256 and 1280?
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/open-compass/VLMEvalKit/issues/1370#issuecomment-3681973980>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAFAUYDMCVR3CFSH4RMIFHT4C7SETAVCNFSM6AAAAACPWOQJOOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMOBRHE3TGOJYGA>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


That makes sense. Thank you for your reply！