Is the Vision model inside the pytorch binaries used ?

#4
by cmp-nct - opened

I followed the python code and it appears that the vit model you have in the tensors is ignored.
in modelling_internlm_xcomposer2.py you build "self.vit = build_vision_tower()" which does that interpolation to the new image size.
However, the model inside the pytorch already seems to have been interpolated beforehand ?
Also the vit loaded seems to be the vanilla openai 336 patch, not the one supplied.

My pytorch isn't superb, maybe I am missing something crucial ?

P.S. Very nice model

Sign up or log in to comment