Electra model from pretrained not loading correctly
Author: hecking
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
 `transformers` version: 4.11.3
- Platform: Linux-5.11.0-36-generic-x86_64-with-glibc2.31
- Python version: 3.9.7
- PyTorch version (GPU?): 1.10.0+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
@LysandreJik

## Information

Model I am using (Electra):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Train an electra model M1
2. Save M1 as pretrained
3. Load model fro pretrained -> M2
4. Make prediction from M1 and M2 -> M2 always gives the same vectors for all tokens

```python
learn = Learner(dls, electra_model,
                loss_func=electra_loss_func,
                opt_func=opt_func ,
                path='./checkpoints',
                model_dir='pretrain',
                cbs=[mlm_cb,
                    RunSteps(c.steps, [0.01, 0.0625, 0.125, 0.25, 0.5, 0.75, 1.0], c.run_name+"_{percent}"),
                     ],
                )
learn.fit(3, cbs=[lr_shedule])

m1 = electra_model.discriminator
m1.save_pretrained("electra-small-test")

m2 = ElectraForPreTraining.from_pretrained("electra-small-test")

tokenizer = ElectraTokenizerFast.from_pretrained("electra-small-generator-test")
tokens = tokenizer.encode('Hello World')
tokens_tensor = torch.tensor([tokens])
```

Getting an embedding from the trained model m1 yields different vectors for each token:

```python
print(m1(tokens_tensor))

Output: 

tensor([[[  2.4823,   0.5022,   1.0171,  ...,   4.2493,   6.4751,  -2.7281],
         [  2.3639,   0.3814,   1.0644,  ...,  -5.1085,   6.1802, -11.2867],
         [  2.2842,   0.5379,   1.1335,  ...,  -4.7094,   5.8275, -10.8865],
         [  2.4677,   1.0243,   1.1420,  ...,  -5.1284,   6.4092, -11.5033],
         [  2.6425,   0.3355,   1.0403,  ...,  -5.5712,   2.2294, -12.2296]]],
       grad_fn=<NativeLayerNormBackward0>)
```

However, using the loaded model m2 all vectors are the same:

```python
print(m2(tokens_tensor))

Output: 

tensor([[[  2.2637,   0.4339,   1.2181,  ...,  -5.1501,   6.0848, -10.8840],
         [  2.2637,   0.4339,   1.2181,  ...,  -5.1501,   6.0848, -10.8840],
         [  2.2637,   0.4339,   1.2181,  ...,  -5.1501,   6.0848, -10.8840],
         [  2.2637,   0.4339,   1.2181,  ...,  -5.1501,   6.0848, -10.8840],
         [  2.2637,   0.4339,   1.2181,  ...,  -5.1501,   6.0848, -10.8840]]],
       grad_fn=<NativeLayerNormBackward0>)
```

Checking the weights of both models all weights are identical.

## Expected behavior

When a model is stored on disk and then loaded again, I would expect exactly the same behaviour.


Comments:
Have you tried doing this?
```
m2 = ElectraForPreTraining.from_pretrained("electra-small-test").discriminator
```

This is not working since I´m saving and loading the discriminator already:

```python
m1 = electra_model.discriminator
m1.save_pretrained("electra-small-test")

m2 = ElectraForPreTraining.from_pretrained("electra-small-test")
```

Ah you are right! This is indeed pretty weird.

I'm not sure what your `electra_model.discriminator` object actually is - I can't seem to find a class that has a `discriminator` attribute anywhere in the docs.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.