Add MVP model
Author: StevenTang1998
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Add MVP models.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@patrickvonplaten 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @StevenTang1998, 

Cool to see a new model here! Do you need help with the failing CI? 

Thanks for your concern. And I really met two issues.
1. I want to add two tokens in the tokenizer. I tried two ways, but they both didn't pass the test.
- add them to the `additional_special_tokens` during tokenizer init.
- use `tokenizer.unique_no_split_tokens` to add them.

2. Another issue is my model didn't pass the test `MvpModelTest.test_beam_sample_generate`, I am a little confused why my model passed other generation tests, while failed in this one. And I didn't understand the motivation of this test, so I don't have some ideas to fix it.

Could you offer some instructions about these issues? Thanks very much!

Hi, @patrickvonplaten. I have fixed the tokenizer issue.

The model issue is
```
E           AssertionError: Lists differ: [[2, 84, 28, 28], [2, 51, 35, 35], [2, 51, 28, 28], [2, 51, 28, 51]] != [[2, 0, 0, 12], [2, 0, 29, 29], [2, 0, 0, 12], [2, 0, 0, 12]]
E           
E           First differing element 0:
E           [2, 84, 28, 28]
E           [2, 0, 0, 12]
E           
E           - [[2, 84, 28, 28], [2, 51, 35, 35], [2, 51, 28, 28], [2, 51, 28, 51]]
E           + [[2, 0, 0, 12], [2, 0, 29, 29], [2, 0, 0, 12], [2, 0, 0, 12]]
```

- As for the model issue, I found the reason:
My model inherits from BART, and I set `forced_bos_token_id` explicitly.
During the generation test (for example in [`_beam_sample_generate`](https://github.com/huggingface/transformers/blob/main/tests/generation/test_generation_utils.py#L423)), the test [`generate`](https://github.com/huggingface/transformers/blob/main/tests/generation/test_generation_utils.py#L440) option will reprocesss the `logits_processor` [here](https://github.com/huggingface/transformers/blob/main/src/transformers/generation_utils.py#L1253). So the `logits_processor` has `ForcedBOSTokenLogitsProcessor`. Whereas, the test [`beam_sample`](https://github.com/huggingface/transformers/blob/main/tests/generation/test_generation_utils.py#L474) option only adds `InfNanRemoveLogitsProcessor` [here](https://github.com/huggingface/transformers/blob/main/tests/generation/test_generation_utils.py#L470).
According to the error result, the first test option generates the `bos_token` (0) in the second position, where the second test option samples the second token.

- Moreover, after I upload files, the failing CI contains errors not related to us.

Could you offer some help to solve them?



Hi @patrickvonplaten, remove specifying `forced_bos_token_id` in the my Config can solve the issue.

However, I still met some issues not related to my model. Could you offer some help?

Hey @StevenTang1998,

Yes indeed the failing tests are not your fault -> could you try to do the same as explained here: https://github.com/huggingface/transformers/pull/17784#issuecomment-1162446039

Hi @patrickvonplaten, thanks for your help! Now I passed all the test, so what is the next step?

Thanks for your comments, I will update it soon.

> 3.) I'd be slighly in favor of renaming all classes to MVPTokenizer and MVPModel because MVP is an acronym the actual name of the paper (but ok for me to leave here as well what do you think @sgugger ?)

Actually disagree rather strongly here ;-) BERT is an acronym and we still use Bert for the model classes. IMO it was a mistake to use all capital GPT, so would prefer keeping Mvp as is!

Hi @patrickvonplaten, thanks very much for your comments.
- We have uploaded our paper in [here](https://github.com/RUCAIBox/MVP/blob/main/paper.pdf), it will be published on ArXiv at Mon, 27 Jun 2022 00:00:00 GMT. We will update the paper url once it announced.
- We have added `# Copied from ...` statements everywhere where applicable.
- Our model can be fine-tuned for sequence classification and question answering (we conducted experiments in our paper), so we reserve them.
- According to the comment of @sgugger, maybe our model name do not need to be changed?
- We have merged conflicts.


Hi @patrickvonplaten, we have update the arxiv paper link.

@sgugger Thanks for your comments! We will fix them following your advice.

@patrickvonplaten @sgugger Thanks for your valuable advice! I have made changes, please review them at your convenience.

Can you just explain why there can't be any Copied from statements for `MvpAttention`, `MvpEncoderLayer` and `MvpDecoderLayer`? The first twos don't have any answer on my comment and the last one as a cryptic "same". Thanks :-)

Hi @sgugger, I'm sorry maybe I didn't press the Comment button yesterday.
We didn't add `Copied from` because we add prompts in these three module. We are a little unclear about the `Copied from` mechanism that we should add them when the code are exactly the same? or almost the same?

Thanks for your explanation! That was the last thing standing, so merging this new model. Thanks a lot for your contribution!

Thanks a lot for your patient comments and guidance!