AttributeError: 'LlamaForCausalLM' object has no attribute 'merge_and_unload'
Author: tshrjn
Pull request: no

### System Info

- `transformers` version: 4.37.0.dev0
- Platform: Linux-5.4.0-155-generic-x86_64-with-glibc2.31
- Python version: 3.10.12
- Huggingface_hub version: 0.20.2
- Safetensors version: 0.4.1
- Accelerate version: 0.26.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When using the TinyLlama model with Lora training as in unsloth's [colab](https://colab.research.google.com/drive/1AZghoNBQaMDgWJpi4RbffGM1h6raLUj9?usp=sharing) shown in readme. On trying to 'merge_and_unload' post training, I get the following error:

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[43], line 1
----> 1 model = model.merge_and_unload()

File /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1695, in Module.__getattr__(self, name)
   1693     if name in modules:
   1694         return modules[name]
-> 1695 raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

AttributeError: 'LlamaForCausalLM' object has no attribute 'merge_and_unload'
```

### Expected behavior

To be able to merge the adaptors.

Comments:
Hi @tshrjn, thanks for raising an issue! 

It looks like this is an issue on how the transformers model is wrapped on unsloth's side. Our model's don't natively have a `merge_and_upload` method. 

@amyeroberts Thanks for taking a look. 

This might be a PEFT library issue. The docs for which states the `merge_and_unload` function [here](https://huggingface.co/docs/peft/v0.7.1/en/package_reference/lora#peft.LoraModel.merge_and_unload).


@tshrjn Thanks for the pointer! cc @younesbelkada 

Hi @tshrjn 
Thanks a lot for the issue! Have you correctly called a `FastLanguageModel.get_peft_model` before running the training? I think that unsloth properly takes care of merging lora weights in the base model if you call `merge_and_unload()` - Can you also print `model` before that error? 
cc @danielhanchen as well

Oh I will check this! I'll run the notebook again to investigate :)

I just updated the Tiny Llama notebook: https://colab.research.google.com/drive/1AZghoNBQaMDgWJpi4RbffGM1h6raLUj9?usp=sharing

Will announce tomorrow, but we now support merging to GGUF / float16 directly. The saving modules are on the very bottom.

But `model.merge_and_unload()` only works if calling `FastLanguageModel.get_peft_model` as Younes described.

I'll investigate the HF notebooks I uploaded as well + will update them with the latest Unsloth release!

Thanks very much @danielhanchen ! 
@tshrjn as stated by @danielhanchen can you confirm running the cell with `get_peft_model` fixes your issue ? 🙏 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

what is the version of Peft library to install for supporting merge_and_uload() method.