Add TF implementation of `XGLMModel`
Author: stancld
Pull request: yes

# What does this PR do?

Fixes #16422

This PR adds TF implementation of `XGLMModel`.

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

@gante @patil-suraj 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16543). All of your documentation changes will be reflected on that endpoint.

Hello @gante, I would like to kindly ask for help here, as I'm not fully sure, where the problem might be :/ Currently, it looks like all the weights are loaded properly by a TF model, but it seems there still must be a glitch :| (I'm still learning TF so I might have made a silly mistake O:] ) Thanks a lot in advance!


> Hello @gante, I would like to kindly ask for help here, as I'm not fully sure, where the problem might be :/ Currently, it looks like all the weights are loaded properly by a TF model, but it seems there still must be a glitch :| (I'm still learning TF so I might have made a silly mistake O:] ) Thanks a lot in advance!

No worries :) Are you trying to run the equivalence test as it is, or are you running another script?

> > Hello @gante, I would like to kindly ask for help here, as I'm not fully sure, where the problem might be :/ Currently, it looks like all the weights are loaded properly by a TF model, but it seems there still must be a glitch :| (I'm still learning TF so I might have made a silly mistake O:] ) Thanks a lot in advance!
> 
> No worries :) Are you trying to run the equivalence test as it is, or are you running another script?

I've tried to run the equivalence tests as they are.

@stancld a little forewarning: I'm having issues in need of debugging in one of my own PRs as well and I'm off tomorrow, so I don't expect to have an answer before next Tuesday :) 

Hey @stancld 👋 Apologies for my delay :) 

I've added two sets of small changes, to hopefully help with test issues:
1. removed `TFCoreModelTesterMixin` from the tests -- it is a slow test suite reserved for key models like `bert` or `gpt2`;
2. added an explicit cast inside a layer -- on my end, I got an exception related to a variable being in `half-precision. On CI, I see that there is no error, only a failed numerical test.

I see that it didn't solve the issue here, even though it is passing on my local env. Digging deeper 🔍 

<img width="1512" alt="Screenshot 2022-04-24 at 15 14 15" src="https://user-images.githubusercontent.com/12240844/164980731-cc0fb174-77f7-4732-b359-d0ac60994c0f.png">

(passing in my env)

I've spun up a machine with a GPU, it also passes if executed on a GPU (a K80). @stancld, does the test fail in your local environment, after the changes? 
- If it fails and you have a GPU, check if passes on CPU (e.g. `CUDA_VISIBLE_DEVICES="" RUN_SLOW=1 RUN_PT_TF_CROSS_TESTS=1 py.test -vv tests/xglm/test_modeling_xglm.py::XGLMModelTest::test_pt_tf_model_equivalence`)
- If it passes, it might be a CI env problem 🤔 

> * RUN_SLOW=1 RUN_PT_TF_CROSS_TESTS=1 py.test -vv tests/xglm/test_modeling_xglm.py::XGLMModelTest::test_pt_tf_model_equivalence

Hi @gante, thanks a lot for the update. I can confirm equivalence tests pass fine on CPU.

It seems equivalence tests are passing after rebasing to the 'main' branch. There are some remaining three failing tests, however, the outcome is a bit random as they don't always fail 🤔 

```
FAILED tests/xglm/test_modeling_tf_xglm.py::TFXGLMModelTest::test_xglm_model_att_mask_past - tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected 'tf.Tensor(False, shape=(), dtype=bool)' to be true. Summarized data: b''
FAILED tests/xglm/test_modeling_tf_xglm.py::TFXGLMModelTest::test_xglm_model_past - tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected 'tf.Tensor(False, shape=(), dtype=bool)' to be true. Summarized data: b''
FAILED tests/xglm/test_modeling_tf_xglm.py::TFXGLMModelTest::test_xglm_model_past_large_inputs - tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected 'tf.Tensor(False, shape=(), dtype=bool)' to be true. Summarized dat...
```

> I can confirm equivalence tests pass fine on CPU.

@stancld have you tried them on GPU? If so, what was the outcome, and what GPU model do you have? I'm trying to figure out why the tests fail in some conditions :D

> > I can confirm equivalence tests pass fine on CPU.
> 
> @stancld have you tried them on GPU? If so, what was the outcome, and what GPU model do you have? I'm trying to figure out why the tests fail in some conditions :D

Haven't tried yet. I'll run tests on GPU later today and let you know what it looks like. :]

@stancld after your most recent push, the equivalence tests passed on CI 🤔 Well, one less problem to consider. 

Hi @gante, I cleaned a test file a little bit to match the other test files. I'm gonna add some integration tests later tonight and then I hope the PR will be ready for the final review :]

Hi @gante, actually not sure which/if any integration tests should be added. Can I ask you, please, to have a look if there's anything missing now?
Also, I've uploaded the TF checkpoint to the HF hub and it should work just fine. You can find a checkpoint [here](https://huggingface.co/Stancld/xglm-564M).

Ah -- you should be able to get rid of the CI errors by rebasing with `main` 👍 

Hello @gante, thanks a lot for your feedback and tips! :] I added an integration test on left padding and found out it actually doesn't work properly and generates some gibberish. Try to dig deep to solve the problem :]

> Hey @stancld 👋 First of all, apologies for my delay in the review 🙈 I think the model is almost ready to go! I've added a few minor comments, including an action point for me (to add the TF weights in the facebook repo)
> 
> Regarding integration tests: a potentially missing test is whether the model can make proper use of left padding (for instance, TFGPT2 was not doing it until recently). You can see an example test [here](https://github.com/huggingface/transformers/blob/main/tests/models/gpt2/test_modeling_tf_gpt2.py#L539)
> 
> As always, thank you for the excellent contribution <3

Hello @gante, is there by chance any MR fixing left-padding for GPT2? O:] 

> Hello @gante, is there by chance any MR fixing left-padding for GPT2? O:]

Check `prepare_inputs_for_generation` in [this](https://github.com/huggingface/transformers/pull/17426) PR (or `main`'s TF GPT2), I suspect TF XGLM is missing `position_ids`. `position_ids` holds the position of the token after accounting for left-padding, allowing the model to get the correct position embedding.

Weights PR open here: https://huggingface.co/facebook/xglm-564M/discussions/1

(one detail needs double-checking, other than that it should be good to go)

Hello @gante, sorry for a bit of delay on this PR. We've been just finishing #16792. I'll try to fix all the issues here afterwards :]

Weights merged -- for the 564M model, the others I will merge after this PR gets merged

(No worries about delays, take your time 👍 )

Hello @gante,

I have updated tests to use TF checkpoints and it looks like there must be a glitch there as some new failing slow tests occurred (`test_batch_generation`, `test_xglm_sample`).

Besides, there's a persisting problem with `test_lm_generate_xglm_left_padding`. And also some peculiar issue with `test_lm_generate_xglm` which raises `ValueError: not enough values to unpack (expected 2, got 1)` inside greedy search.

See all errors below.

```
FAILED tests/models/xglm/test_modeling_tf_xglm.py::TFXGLMModelTest::test_batch_generation - AssertionError: Lists differ: ['Hel[17 chars]ittle bit of a shy one, but he is very friendl[65 chars]ngs'] != ['Hel[17 chars]ittleተኛውowany Gami...
FAILED tests/models/xglm/test_modeling_tf_xglm.py::TFXGLMModelLanguageGenerationTest::test_lm_generate_xglm - ValueError: not enough values to unpack (expected 2, got 1)
FAILED tests/models/xglm/test_modeling_tf_xglm.py::TFXGLMModelLanguageGenerationTest::test_lm_generate_xglm_left_padding - AssertionError: 'Toda[19 chars]y andიას Nomເຮົາ trẻ napríklad სამართლ Bouleva[24 chars]іnye' != 'Toda[19 chars]y ...
FAILED tests/models/xglm/test_modeling_tf_xglm.py::TFXGLMModelLanguageGenerationTest::test_xglm_sample - AssertionError: 'Toda[14 chars]y and벗5huhuxhuman яркиیارە daerah opisujelanıb[27 chars]ågan' != 'Toda[14 chars]y and the sun is s...
=========================================================================================== 4 failed, 31 passed, 1 skipped, 81 warnings in 58.29s ============================================================================================
```

Try to investigate these problems more deeply this week :]

Ola @gante, I'm pleased to announce that all failing slow tests have been fixed and work fine now (including left-padding generation).

However, there's a problem with a `tf_model.h5` in the `facebook/xglm-574M`, as the model using those weights produces gibberish output. I can confirm, nonetheless, that everything works perfectly with my TF weights available in [`Stancld/xglm-564M](https://huggingface.co/Stancld/xglm-564M). 

cc: @patil-suraj @Rocketknight1 

> However, there's a problem with a tf_model.h5 in the facebook/xglm-574M, as the model using those weights produces gibberish output. I can confirm, nonetheless, that everything works perfectly with my TF weights available in [`Stancld/xglm-564M](https://huggingface.co/Stancld/xglm-564M).

😬  I'll check what went wrong with my conversion. You converted using the `from_pretrained(from_pt=True)`, then stored the weights with `.save_weights()`, correct?

> > However, there's a problem with a tf_model.h5 in the facebook/xglm-574M, as the model using those weights produces gibberish output. I can confirm, nonetheless, that everything works perfectly with my TF weights available in [`Stancld/xglm-564M](https://huggingface.co/Stancld/xglm-564M).
> 
> 😬 I'll check what went wrong with my conversion. You converted using the `from_pretrained(from_pt=True)`, then stored the weights with `.save_weights()`, correct?

I used `.save_pretrained(...)` instead of `.save_weights().`

@stancld the conversion I pushed was built with the first version of the `pt-to-tf` CLI, which was not converting the model head. I've opened a PR with the updated weights, which also contain the model head 🙌 

There is still one minor problem, but it's outside the scope of this PR -- the PT weights are stored in `float16`, and our TF tools do not have support for half-precision (yet!). 

weights PR 👉  https://huggingface.co/facebook/xglm-564M/discussions/2

They were merged -- can you check whether the weights work? :)

> They were merged -- can you check whether the weights work? :)

Yes, can confirm all tests (including slow ones) work!

Awesome, so all that's left is a full review.

@Rocketknight1 @patil-suraj 

@gante Finally got there, rebased to the current main and add the attribute :]

Needs to be rebased with `main` again, the problems in CI were already fixed there :)

Going to ask @patil-suraj for a review, then we can merge it (and convert the remaining models!)

@gante @patrickvonplaten One failing test fixed :]

Splendid! Thank you for all the work, @stancld 🧡 

I'm going to merge the PR and push the weights for the other XGLM architectures 🔥