Repository: huggingface/transformers
Pull request: fix "Cannot copy out of meta tensor; no data!" issue for BartForConditionalGeneration model

fix issue: https://github.com/huggingface/transformers/issues/36247

cause of this issue:

[bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn/tree/main)  checkpoint only has `model.decoder.embed_tokens.weight` while `load_state_dict`, but in BartModel implementation, both `model.decoder.embed_tokens.weight` and `model.encoder.embed_tokens.weight` tie to the `shared.weights` which doesn't have corresponding real weights in checkpoints, so when `tie_weights` are called, things are back to a void `shared.weights` which in `meta` device, then lead to the model.to issue

@SunMarc @Rocketknight1 , I don't know whether this PR meet your standard since it changed modeling code, pls let me know your thoughts, thx.


Changed files:
filename: src/transformers/models/bart/modeling_bart.py
patch: @@ -1442,8 +1442,15 @@ def __init__(self, config: BartConfig):
 
     def _tie_weights(self):
         if self.config.tie_word_embeddings:
-            self._tie_or_clone_weights(self.encoder.embed_tokens, self.shared)
-            self._tie_or_clone_weights(self.decoder.embed_tokens, self.shared)
+            # Some model checkpoints like "facebook/bart-large-cnn"'s embedding weight is in decoder.embed_tokens, need check here, see issue #36247
+            if self.shared.weight.device == torch.device(
+                "meta"
+            ) and self.decoder.embed_tokens.weight.device != torch.device("meta"):
+                self._tie_or_clone_weights(self.encoder.embed_tokens, self.decoder.embed_tokens)
+                self._tie_or_clone_weights(self.shared, self.decoder.embed_tokens)
+            else:
+                self._tie_or_clone_weights(self.encoder.embed_tokens, self.shared)
+                self._tie_or_clone_weights(self.decoder.embed_tokens, self.shared)
 
     def get_input_embeddings(self):
         return self.shared
@@ -1599,6 +1606,11 @@ def get_output_embeddings(self):
     def set_output_embeddings(self, new_embeddings):
         self.lm_head = new_embeddings
 
+    def _tie_weights(self):
+        if self.config.tie_word_embeddings:
+            self.model._tie_weights()
+            self._tie_or_clone_weights(self.lm_head, self.model.shared)
+
     @add_start_docstrings_to_model_forward(BART_INPUTS_DOCSTRING)
     @replace_return_docstrings(output_type=Seq2SeqLMOutput, config_class=_CONFIG_FOR_DOC)
     @add_end_docstrings(BART_GENERATION_EXAMPLE)

filename: src/transformers/models/bigbird_pegasus/modeling_bigbird_pegasus.py
patch: @@ -2479,6 +2479,11 @@ def get_output_embeddings(self):
     def set_output_embeddings(self, new_embeddings):
         self.lm_head = new_embeddings
 
+    def _tie_weights(self):
+        if self.config.tie_word_embeddings:
+            self.model._tie_weights()
+            self._tie_or_clone_weights(self.lm_head, self.model.shared)
+
     @add_start_docstrings_to_model_forward(BIGBIRD_PEGASUS_INPUTS_DOCSTRING)
     @replace_return_docstrings(output_type=Seq2SeqLMOutput, config_class=_CONFIG_FOR_DOC)
     @add_end_docstrings(BIGBIRD_PEGASUS_GENERATION_EXAMPLE)