ydshieh
/

kosmos-2-patch14-224

Feature Extraction

Model card Files Files and versions Community

ydshieh HF staff commited on Sep 1, 2023

Commit

43ed35e

•

1 Parent(s): b9379a4

Update modeling_kosmos2.py

Files changed (1) hide show

modeling_kosmos2.py +1 -1

modeling_kosmos2.py CHANGED Viewed

@@ -1007,7 +1007,7 @@ class Kosmos2TextTransformer(nn.Module):
             inputs_embeds = self.embed_tokens(input_ids)
         if img_features is not None:
-            inputs_embeds[img_input_mask.to(dtype=torch.bool)] = img_features
         inputs_embeds = inputs_embeds * self.embed_scale

             inputs_embeds = self.embed_tokens(input_ids)
         if img_features is not None:
+            inputs_embeds[img_input_mask.to(dtype=torch.bool)] = img_features.view(-1, img_features.size(-1))
         inputs_embeds = inputs_embeds * self.embed_scale