BartForCausalLM analogs to `ProphetNetForCausalLM`
Author: sadakmed
Pull request: yes

# What does this PR do?

Implementing BartForCausalLM anologs for ProphetNetForCausalLM

Fixes #9066 


Comments:
Hy @sadakmed, 

let me know if you need help on the issue or if you don't find the time to tackle it. I'll then just make it open to the "public" again :-) 

@patrickvonplaten  The loss function it what I stuck on, thank you very much for your guidance.


> Let me know if you need help or are stuck :-)

for sure I will ;-) 

Hey @sadakmed,

Do you have an update on the PR? It's been three weeks now and it would be great to merge this soon. Sorry, we're very fast-moving in this lib and other community contributors have started asking for this feature. By next week, I'll probably have to take a look myself or redistribute the issue. 

Hi @patrickvonplaten  my apologies, 

Could you Please see it now, lemme know if anything is missing.  

Hi @patrickvonplaten, working on the test 'BartStandaloneCausalLM': I dont know if the `self.model_tester` in 'setUp' should be 'BartDecoderTester' (needed to be implemented), or do u recommend something else. 

 

Hey @sadakmed, 

Thanks a lot for you additions here :-) 
Yes, we need a new `BartStandaloneDecoderModelTester` analog to how it's done for ProphetNet in `tests/test_modeling_prophetnet.py`. Do you want to give it a try? Otherwise, I can go into your PR and see how to add the tests :-) 

Hi @patrickvonplaten 

> Do you want to give it a try?

of course, I'm working on it, 

Hi @patrickvonplaten, I just pushed the test, could you please check it out! 
thaaaanks 

Hey @sadakmed, 

I corrected `BartForCausalLM` and also added `MBartForCausalLM`. It would be awesome if you could take care of adding `MarianForCausalLM`, `PegasusForCausalLM`, `BlenderbotForCausalLM`, and `BlenderbotSmallForCausalLM`. 

To do so you can simply copy everything that was done for `MBart` in this PR 1-to-1 to the mentioned models above. Let me know if sounds feasible for you :-) 

Thanks a lot for your help so far!

> adding `MarianForCausalLM`, `PegasusForCausalLM`, `BlenderbotForCausalLM`, and `BlenderbotSmallForCausalLM`.

implementing it for use with EncoderDecoder or just the test? 

Yes I would like to do it, with all pleasure.

> > adding `MarianForCausalLM`, `PegasusForCausalLM`, `BlenderbotForCausalLM`, and `BlenderbotSmallForCausalLM`.
> 
> implementing it for use with EncoderDecoder or just the test?
> 
> Yes I would like to do it, with all pleasure.

For those models, there is no need to add a test to `EncoderDecoderModel`. We should only copy-paste the code that was added to MBart to those models and also copy-paste the test in `test_modeling_marian.py` e.g.

@patrickvonplaten  Could you please check the test if it well, and about the test of `Decoder only` I didn't get what do you mean!!

It would be nice to fix the tests and also add tests for `Pegasus`, `Blenderbot`, and `BlenderbotSmall`

> It would be nice to fix the tests and also add tests for `Pegasus`, `Blenderbot`, and `BlenderbotSmall`

@patrickvonplaten, exactly like the one was for Marian?

@patrickvonplaten  could you check please!

**UPDATE:**

@LysandreJik @sgugger 
This PR enables all Bart-like models to be used in combination with the Encoder-Decoder framework. The model `BartForCausalLM` is added for Bart and then copied to all other models via the copying mechanism. Also, a new model tester is added for all those models. 
While working on this I found a small bug for a very edge-case scenario for Bart and corrected it here: https://github.com/huggingface/transformers/pull/9128/files#r569436360 . The newly added tests were failing, which made me aware of the bug. 
Also, I had to slightly change the `check_repo.py` file so that it counts both classes from `all_model_classes` with 1 and 2 paratheses. 


Great job @sadakmed 

> Great job @sadakmed

wouldn't happen without you, thank you very much.  see u in the next PR ;)