Add support for SciBART by UCLANLP
Author: xiaowu0162
Pull request: yes

# What does this PR do?

Add the support for the SciBART model (https://arxiv.org/abs/2212.10233). This is a BART model trained from scratch on the S2ORC corpus. Its tokenizer is a sentencepiece tokenizer trained from scratch. This PR supports using the newly trained tokenizer. The model checkpoints are already uploaded to https://huggingface.co/uclanlp/scibart-base and https://huggingface.co/uclanlp/scibart-large.

Implementation-wise, this PR refers to #1839.

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker and @younesbelkada


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_23353). All of your documentation changes will be reflected on that endpoint.

Hi @younesbelkada, this pr is ready for review.

Thank you for the comments! I have fixed the mentioned issues and they are ready for review. @amyeroberts @younesbelkada 

> Could you explain the main difference that are added in this tokenizer? I am not sure we have to add a new class for it, that's why I am asking

Yes. We are adding a new model pre-trained from scratch on the science corpus. This model has exactly the same architecture as BART but with a different vocabulary. The new tokenizer class is needed because our tokenizer is trained with sentencepiece, while the facebook BART model does not use sentencepiece. Does this solve your concerns? @ArthurZucker 

Sure, but we have bunch of already implemented tokenizers that rely on `spm`, with slow to fast converters. Look at the `BarthezTokenizer` for example, code seems kinda duplicate. `XGLM` looks also very similar, same for `XLM_Roberta`. 

> Sure, but we have bunch of already implemented tokenizers that rely on `spm`, with slow to fast converters. Look at the `BarthezTokenizer` for example, code seems kinda duplicate. `XGLM` looks also very similar, same for `XLM_Roberta`.

@ArthurZucker Thank you for the reply. `BarthezTokenizer` is indeed similar (so is `XGLMTokenizer`). However, I believe it has some different assumptions and cannot be directly used by SciBART. For example, the default tokens 0, 1, 2, 3 are different (https://github.com/huggingface/transformers/blob/b7b729b38d12309185bcc9fdf8b55418a1ad2421/src/transformers/models/barthez/tokenization_barthez.py#L160). Letting SciBartTokenizer inherit from it also does not seem to make sense because it breaks the modularity. 

What are the actionable items here? Our priority is to enable users to use the SciBART model.

Sorry for being noisy 😅 If you are going to make the model available on the hub, and the only differences are the default tokens, I would simply recommend you to have the tokenizer on the hub. We can't accept a new model which only has this one line that differs. An other solution is to open a PR to allow this default `fairseq_token_ids` to be an argument of the init, which would allow you to store it in the tokenizer config to easily use the BarthezTokenizer! 
Another way is to hold your code on the hub! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.