Fix Failed tests with mobile bert resize tokens embedding
Author: abuelnasr0
Pull request: yes

Fixes the failures introduced by #33325

The tests failed with mobilebert because of a missing transposing for the `old_lm_head`. This PR fixes that. I have tried the two failed tests locally.
It's weird that all tests passed before merging. EDIT: I see now, some tests were skipped

I have also changed the logic when the covariance matrix is not positive definite, just initialize the new embeddings with the mean if covariance is not positive definite.

c.c. @ArthurZucker



Comments:
Ah could you update `tests/models/recurrent_gemma/test_modeling_recurrent_gemma.py::RecurrentGemmaModelTest::test_resize_tokens_embeddings ` as welll? 

Ah and `tests/models/git/test_modeling_git.py::GitModelTest::test_resize_tokens_embeddings - AssertionError: Padding_idx must be within num_embeddings` as well 

See here 😅 https://app.circleci.com/pipelines/github/huggingface/transformers/106584/workflows/539a0425-e7e1-4a82-aedb-d7134aa524ba/jobs/1415704

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33950). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker I have addressed mobilebert test, GitModeltest, and recurrent_gemma test.

recurrent_gemma test failed because an outlier was sampled, so I multiplied covariance by 1e-9 instead of 1e-5. 
Git model tests failed because the config was overwritten after the first resizing test and used to initialize the model again, so I created a new copy for the new model initialization. 

I feel bad about those tests' failures actually, I wanted to deliver good code but tests didn't help me. 😅

The outlier of recurrent_gemma got sampled only after merging the code haha.

And I am not sure if other tests were actually skipped before merging. It's weird.
Do you know why the test failures appeared after merging? mobilebert and GitModeltest?

Yep! They are not part of the important model, the test fetcher seems to badly behave! It should have found out the whole dependencies! 

No worries, we are the ones who set you up for failure in that case! 

cc @ydshieh if you can have a look at the reasons why this was not fetched when you have time! 

> Yep! They are not part of the important model, the test fetcher seems to badly behave! It should have found out the whole dependencies!
> 
> No worries, we are the ones who set you up for failure in that case!
> 
> cc @ydshieh if you can have a look at the reasons why this was not fetched when you have time!

Could one of you provide a link of the previous (PR) job run page where you believe there is something being missed by the test fetcher?

Yes! https://app.circleci.com/pipelines/github/huggingface/transformers/106469/workflows/2ce53b8a-9a18-4e16-9b94-fd3b786c0916/jobs/1414187 is all green, but a few hours later: https://app.circleci.com/pipelines/github/huggingface/transformers/106724/workflows/fb68022e-2d64-44e9-91a5-f86c18f73cdd/jobs/1417162 or any other test torch on main would fail!

In the mean time @abuelnasr0 could you commit with a message `test_all` 🤗 

This way all models should be ran! 

<img width="1048" alt="image" src="https://github.com/user-attachments/assets/9866b094-3777-450d-a7fe-ecac56378a7f">
A few failing tests still! 

``` RUN_SLOW=1 pytest  -n 4 tests/models/*/test_modeling_* -k test_resize_tokens_embeddings``` to filter!

I hope you don't mind @abuelnasr0 -- this PR is blocking other PRs, so I'm taking care of the rest of the fixes 🤗 

(@ArthurZucker -- all tests in `pytest tests/models/ -k test_resize_tokens_embeddings` were green)

@gante No problem. That is completely fine.
I am sorry for blocking other PRs! Thanks for the fixes!

@gante thanks for fixing it!