'transformer_model' object has no attribute 'module'
Author: sqinghua
Pull request: no

### System Info

```shell
Platform: Kaggle
python: 3.7
torch: 1.13.0
transformers: 4.27.4
tensorflow: 2.11.0
pre-trained model used: XLNET
```


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I reproduce others' open-sourced work on Kaggle. Following the partly code:
``` 
import torch  
from torch import nn 
from transformers import XLNetConfig, XLNetLMHeadModel, XLNetModel, XLNetTokenizer,AutoTokenizer

class transformer_model(nn.Module):
  def __init__(self, model_name, drop_prob = dropout_prob):
    super(transformer_model, self).__init__()
    configuration = XLNetConfig.from_pretrained(model_name, output_hidden_states=True)
    self.xlnet = XLNetModel.from_pretrained(model_name, config = configuration) 
    ...
```


```
def train(model,  optimizer, scheduler, tokenizer, max_epochs, save_path, device, val_freq = 10):
  
  bestpoint_dir = os.path.join(save_path)
  os.makedirs(bestpoint_dir, exist_ok=True)

  ...
  model.save_pretrained(bestpoint_dir)    #here
  print("Saving model bestpoint to ", bestpoint_dir)
  ...



model = transformer_model(model_name).to(device)
...
```

Error messages: 'transformer_model' object has no attribute 'save_pretrained'
```
'transformer_model' object has no attribute 'save_pretrained'
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_23/1390676391.py in <module>
    414                                             num_training_steps = total_steps)
    415 
--> 416 train(model,  optimizer, scheduler, tokenizer, epochs, save_path, device)
    417 
    418 print(max_accuracy, "\n", max_match)

/tmp/ipykernel_23/1390676391.py in train(model, optimizer, scheduler, tokenizer, max_epochs, save_path, device, val_freq)
    385 
    386             # To save the model, uncomment the following lines
--> 387             model.save_pretrained(bestpoint_dir)
    388             print("Saving model bestpoint to ", bestpoint_dir)
    389 

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in __getattr__(self, name)
   1264                 return modules[name]
   1265         raise AttributeError("'{}' object has no attribute '{}'".format(
-> 1266             type(self).__name__, name))
   1267 
   1268     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

AttributeError: 'transformer_model' object has no attribute 'save_pretrained'
```

### Expected behavior

```shell
`model.save_pretrained(...) `should work, I tried to fix the problem by using `model.module.save_pretrained(...)` and `torch.save(...)`, but failed.
How can I fix the problem? Thx!
```


### Checklist

- [X] I have read the migration guide in the readme. ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers); [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [X] I checked if a related official extension example runs on my machine.

Comments:
@sqinghua, this is happening because the class `transformer_model` inherits from PyTorch's nn.Module class, which doesn't have a `save_pretrained` method. `save_pretrained` is a transformers library specific method common to models which inherit from `PreTrainedModel`. 

I'm not sure why `torch.save(...)` doesn't work. I would need more information to be able to help e.g. traceback and error message to know if it's a transformers related issue. 

It should be possible to save the xlnet model out using `model.xlnet.save_pretrained(checkpoint)`. This won't save out any additional modeling which happens in `transformer_model` e.g. additional layers or steps in the forward pass beyond passing to `self.xlnet`. 

@amyeroberts  Thank you for your detailed answer, it was very helpful. :)