Add TF implementation of LongT5
Author: stancld
Pull request: yes

Fixes #18063 

Add:
- [ ] Fix PT-TF equivalence (Local)
- [ ] Fix PT-TF equivalence (TGlobal)
- [ ] Run all slow tests
- [ ] Prepare TF checkpoints
  - [long-t5-local-base](https://huggingface.co/Stancld/long-t5-local-base)
  - [long-t5-local-large](https://huggingface.co/Stancld/long-t5-local-large)
  - [long-t5-tglobal-base](https://huggingface.co/Stancld/long-t5-tglobal-base)
  - [long-t5-tglobal-large](https://huggingface.co/Stancld/long-t5-tglobal-large)
  - long-t5-tglobal-xl https://github.com/huggingface/transformers/issues/19965

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18812). All of your documentation changes will be reflected on that endpoint.

Hi @patrickvonplaten and @gante,
FYI I've been gradually fixing some PT-TF discrepancies -- I should have some spare time again next weekend, so hopefully, then it should be ready for review :]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@stancld should I reopen the PR? :)

Hi @gante, yes, I'd open that again.

I apologize for being so slow here, but I've been pretty busy now. I'll try to finish this.

No worries @stancld, take your time 🤗 And thank you for working on it!

Hi @gante, I managed to fix some bugs. There are still some minor discrepancies between PT and TF implementations. Would you mind having a first look if you spot any obvious differences, please? :]

Otherwise, TF-only tests seem to be passing 🐰 

(Btw, CI is passing, but the tests are failing locally, so I'm not really sure :D )

@stancld Will have a look 👍 Can I have a copy of the error(s) you see locally? (I'm assuming on the slow tests)

Also cc @ArthurZucker here

> @stancld Will have a look 👍 Can I have a copy of the error(s) you see locally? (I'm assuming on the slow tests)

Sorry for the late reply. I fail on `PT-TF` equivalence tests, basically, saying there's a too high difference between outputs.

Hey @stancld ! Thanks for the addition! There are a few approaches we can take here. Sometimes the tolerance is a bit too high and part of the hidden states don't match but the final output does, in that case, we can lower the tolerance (maybe to around `4e-2` other wise, I will have a look ! 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18812). All of your documentation changes will be reflected on that endpoint.

Almost there I think :-) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.