Add OFA configuration, tokenizer and modeling
Author: JustinLin610
Pull request: yes

# What does this PR do?

Fixes #15813 and #16265. Added OFA configuartion, tokenizer and Modeling. 

## Who can Review? 

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

cc @patil-suraj @gchhablani


Comments:
Thank you so much @JustinLin610 for adding this PR. Sorry I hadn't had the chance to work on it so far.

@patil-suraj If anything is needed from my end please let me know.

I can address the failing tests. Please give me until today.

> I can address the failing tests. Please give me until today.

Thanks very much for your help! That would be very nice for me cuz I'm not familiar with these things...

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17109). All of your documentation changes will be reflected on that endpoint.

> I can address the failing tests. Please give me until today.

I just passed the tests (mainly making the test case simpler), but I am not sure if there is any other things that I should do. If you have time, feel free to shoot me some advice on how to improve the code:)

Thanks a lot for the PR @JustinLin610 ! I will review this PR either today or tomorrow. LMK if you have any questions, happy to help :)

> Thanks a lot for the PR @JustinLin610 ! I will review this PR either today or tomorrow. LMK if you have any questions, happy to help :)

Great! Looking forward to your reply!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

> Thanks a lot for the PR @JustinLin610 ! I will review this PR either today or tomorrow. LMK if you have any questions, happy to help :)

I have just sync my branch with main. Any ideas or comments for my PR?  btw it seems that the testing of big bird flax causes the breakdown of the whole testing...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

pinging @gchhablani @patil-suraj 
@LysandreJik 

really want this cool model, merge in huggingface :)

@LysandreJik @patil-suraj Please let me know if anyone else is working on this, otherwise I can give it another attempt.

This is a nice feature. Looking forward to it.

> @LysandreJik @patil-suraj Please let me know if anyone else is working on this, otherwise I can give it another attempt.

Still on this..., and also one of my colleagues will participate in this. Would you mind if you and Patil check the code and give me some feedback?

Is it possible to re-open this PR so it may get attention?

> Is it possible to re-open this PR so it may get attention?

Good suggestion. Maybe I should delete the branch and open a new PR, as my branch has a lot of diffs with the main branch as time passed, and thus I have to merge a lot of PRs for synchronization... I'll update to our latest version first and add more test cases for merging... 

If you are interested, you can check our official repo and the specific branch for transformers. We have confirmed that this version can reach the same performance as the official code based on Fairseq. https://github.com/OFA-Sys/OFA/tree/feature/add_transformers

gently tagging the great @NielsRogge in case they might have a chance to look at this one :-) OFA is quite strong and it would be amazing to have first-party huggingface support for it