Question about bart-base model
Author: HiXiaochen
Pull request: no

When I use 'bart-base', I just want to update some parts of the parameters, so I call function"model.named_parameters()", and let "requires_grad" of parameters which I want to update be true, and others be false. However, when I call "model.state_dict()", I found "model.encoder.embed_tokens.weight", "model.decoder.embed_tokens.weight", "lm_head.weight" are both appear in state_dict() but not in named_parameters(). However, after I checked the initialization code of BartModel, I found all these weights are initialized with nn.Embedding() or nn.Parameters():

'''
self.shared = nn.Embedding(vocab_size, config.d_model, padding_idx)
self.encoder = BartEncoder(config, self.shared)
self.decoder = BartDecoder(config, self.shared)
'''

'''
self.lm_head = nn.Linear(config.d_model, self.model.shared.num_embeddings, bias=False)
'''
So all of they could be updated during training theoretically.Why don't they appear in mode.named_parameters()?

Comments:
`BartModel` itself doesn't have a language modeling head, only `BartForConditionalGeneration` does. The latter adds a language modeling head on top of `BartModel`.

> `BartModel` itself doesn't have a language modeling head, only `BartForConditionalGeneration` does. The latter adds a language modeling head on top of `BartModel`.

Thanks for your reply!!
Sorry that my description is not clear. The model  I used is BartForConditionalGeneration. As I described above, "model.encoder.embed_tokens.weight", "model.decoder.embed_tokens.weight","lm_head.weight" , as well as"final_logits_bias" appear in model.state_dict() but not in model.named_parameters(). I know that "final_logits_bias" is registered in model.buffers(), so it's normal. But aren't the other three supposed to be trainable in downstream missions(which means they should be in model.parameters())? 

That's because input and output embeddings are tied (i.e. shared). This can be verified by printing the named parameters:

```
from transformers import BartForConditionalGeneration

model = BartForConditionalGeneration.from_pretrained("facebook/bart-base")

for name, param in model.named_parameters():
  print(name, param.shape)
```

which prints:
```
model.shared.weight torch.Size([50265, 768])
(...)
```

You can also verify that the weights of the embed_tokens and lm_head for example are exactly the same, like so:

```
import torch

assert torch.allclose(model.model.encoder.embed_tokens.weight, model.lm_head.weight)
```

> That's because input and output embeddings are tied (i.e. shared). This can be verified by printing the named parameters:
> 
> ```
> from transformers import BartForConditionalGeneration
> 
> model = BartForConditionalGeneration.from_pretrained("facebook/bart-base")
> 
> for name, param in model.named_parameters():
>   print(name, param.shape)
> ```
> 
> which prints:
> 
> ```
> model.shared.weight torch.Size([50265, 768])
> (...)
> ```
> 
> You can also verify that the weights of the embed_tokens and lm_head for example are exactly the same, like so:
> 
> ```
> import torch
> 
> assert torch.allclose(model.model.encoder.embed_tokens.weight, model.lm_head.weight)
> ```

I got it. Thanks very much for your patient answer!!!