Is the Vision model inside the pytorch binaries used ?
#4
by
cmp-nct
- opened
I followed the python code and it appears that the vit model you have in the tensors is ignored.
in modelling_internlm_xcomposer2.py you build "self.vit = build_vision_tower()" which does that interpolation to the new image size.
However, the model inside the pytorch already seems to have been interpolated beforehand ?
Also the vit loaded seems to be the vanilla openai 336 patch, not the one supplied.
My pytorch isn't superb, maybe I am missing something crucial ?
P.S. Very nice model