Potential bug when batch inferencing with left padding.

#26
by Snorlax - opened

In line 213 of modeling_minicpmv.py, after performing left padding on input_ids, the image_bound is still using the result from before the left padding. This seems to cause image features to be put to the wrong position in line 153).

OpenBMB org
β€’
edited Jun 21

We haven't tested batch inference yet. This is indeed wrong during batch inference. You need to fix the image_bound here and pass in a correct attention_mask.
Or you can reset the image_bound by special token

Sign up or log in to comment