compute_loss in trainer failing to label shift for PEFT model when label smoothing enabled.
Author: njbrake
Pull request: yes

# What does this PR do?

When training LlamaForCausalLM with PEFT enabled, I noticed that the compute_loss function of the trainer with label_smoothing enabled was not shifting the labels. Further investigation found that `unwrap_model(model)._get_name()` returned "PeftModelForCausalLM", and the `MODEL_FOR_CAUSAL_LM_MAPPING_NAMES` dict doesn't have that, so the label shift was not happening. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

I believe this PR would be a fit for @sgugger review. 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
No we cannot add this like that, as this is not a model that lives in Transformers.
cc @younesbelkada 

@njbrake thanks for the PR, can you elaborate more on the label shifting issue? 
https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L832 should be automatically shifting the labels if you pass a dataset with labels 

@sgugger I think your comment makes sense. This issue is showing up at https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L2680

In that case, it seems to make sense to me that in this context, the trainer should see the model that is sitting "behind" the PEFT model, so that it would see that the PEFT model was the LLama architecture. i'm not sure on whether that means a change is required in the PEFT library, the trainer code, or in my code (or maybe in all three 😄 )

@younesbelkada Since i'm using the label smoother, the trainer API pops out the labels via https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py#L2669 before calling the forward function.

I don't think we should unwrap the model from its peft container, as it would trigger bug when saving (Younes will confirm!) so the best fix is probably to add an or clause in the specific test in the Trainer.

Thanks for clarifying @njbrake that makes sense, I second what @sgugger said, a better fix would be to add a proper check to see if the model is and instance of peft model on the appropriate line, otherwise users might encounter unexpected behaviours (e.g. when saving the model)

_The documentation is not available anymore as the PR was closed or merged._