ccdv
/

lsg-distilbert-base-uncased-4096

Model card Files Files and versions Community

ccdv commited on Aug 28, 2023

Commit

93a4f16

•

1 Parent(s): d766a2f

fix dtype

Files changed (1) hide show

modeling_lsg_distilbert.py +1 -1

modeling_lsg_distilbert.py CHANGED Viewed

@@ -766,7 +766,7 @@ class LSGTransformer(Transformer):
         return_dict: Optional[bool] = None,
     ) -> Union[BaseModelOutput, Tuple[torch.Tensor, ...]]:  # docstyle-ignore
-        attn_mask = attn_mask.float()
         mask_value = 0
         n, t = attn_mask.size()

         return_dict: Optional[bool] = None,
     ) -> Union[BaseModelOutput, Tuple[torch.Tensor, ...]]:  # docstyle-ignore
+        attn_mask = attn_mask.to(dtype=x.dtype)
         mask_value = 0
         n, t = attn_mask.size()