[WIP] Add GeoV model
Author: vpj
Pull request: yes

# What does this PR do?

This PR adds the 9B parameter GeoV language model trained by Georges Harik.

Comments:
cc @ArthurZucker 

[Model weights GeoV/GeoV-9b](https://huggingface.co/GeoV/GeoV-9b)

Hey @vpj feel free to ping me for any guidance ! 😉 Also if you need a review tell me

Yeah need a review. Im new to huggingface transformers. Just went by the tutorials. Let me know what else needs to be done in order to merge this. Thanks

@ArthurZucker 

Sure ! Reviewing now! 

@ArthurZucker I pushed a bunch of changes and replied to your comments. Can you please take a look. Thanks

Sure! 
Can you also follow the instruction in the failures of the CI ( for example add `geoV` to the `toctree.yml` and running `make style` to reformat the files) ? Also the `tests_tf` seems to be failing because of the import of `GeoVForCausalLM`. It should be protected let me check

I ran `make style` and thats what changes that assert statement in the reformer. It didn't do any changes to geov code.

```
FAILED tests/models/whisper/test_modeling_flax_whisper.py::FlaxWhisperModelTest::test_equivalence_pt_to_flax - AssertionError: 1.1205673e-05 not less than or equal to 1e-05 : outputs.encoder_last_hidden_state: Difference between PyTorch and Flax is 1.1205673217773438e-05 (>= 1e-05).
```

This is why torch_and_flax test is failing

@ArthurZucker  Very much appreciate the help so far. Can you please help me get this PR ready by tomorrow since I won't be available for a week after tomorrow. Thank you

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22403). All of your documentation changes will be reflected on that endpoint.

```
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_no_dist - TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
FAILED tests/models/pix2struct/test_image_processing_pix2struct.py::Pix2StructImageProcessingTest::test_expected_patches - PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x7f2bc8801950>
```

This error for tests_torch

These two tests seem unrelated to your PR, pull from main and normally they should dissappear 

Ok the history of the PR got a little bit messed up 😅 it's alright it can happen from time to time! You can either rebase on main starting from `[4bd65f3](https://github.com/huggingface/transformers/pull/22403/commits/4bd65f354c8366feccae95278c1f0b3a85110b3b)` (for example as it is one commit unnafected) or you can do a soft reset to your first commit, add only your modifications and force push. And then pull from main 

The styling depends on the version of `black` that you are using. Seems like most of the test are now good, the last should work with a `pip install black==23.1` 

Yeah messed up by doing a rebase instead of a merge

I am not sure what the check means by imports order/format, to me it looks quite similar to other files.

Ok this is ruff acting up, I use `ruff 0.0.258` ( we recently pinned the correct one)

What should I do? Do I have to install `ruff 0.0.258` and run `make style`?

Oh thanks, didn't know that. I saw this https://huggingface.co/docs/transformers/add_new_model and thought I had to create a pull request.

So, just to make sure I'm clear, should I close this PR and share the model according to https://huggingface.co/docs/transformers/custom_models?

Yes! It would be the best 😉 Thanks for your comprehension! 

Just out of curiosity, how do you choose which models to add to the repo and what goes to the hub?

Added to the hub, but it doesn't work with pipelines (text-generation).

How do I register the model for `text-generation`.

This is what I'm doing now

```
  GeoVConfig.register_for_auto_class()
  GeoVModel.register_for_auto_class("AutoModel")
  GeoVForCausalLM.register_for_auto_class("AutoModelForCausalLM")
  GeoVTokenizer.register_for_auto_class()
```

Trying to load the pipeline with
```
generator = pipeline(model="GeoV/GeoV-9b", trust_remote_code=True)
```
gives the error

```
The model 'GeoVForCausalLM' is not supported for text-generation. Supported models are ...
```

Thanks

> Just out of curiosity, how do you choose which models to add to the repo and what goes to the hub? 

The more we grow, the more we are trying to add models to the hub! Especially if the model does not have a lot of changes compared to a model that we already support!

For the issue, I think you have to update the mapping `MODEL_FOR_MASKED_LM_MAPPING` by adding your model

How can I change `MODEL_FOR_MASKED_LM_MAPPING` if I'm adding to the hub?

The same way you did for the `AUTO_CONFIG_MAPPING`.
An example from [here](https://huggingface.co/THUDM/glm-2b/blob/main/config.json): 
```python 
config.json:
...
"auto_map": {
  "AutoConfig": "configuration_glm.GLMConfig",
  "AutoModel": "modeling_glm.GLMModel",
  "AutoModelForSeq2SeqLM": "modeling_glm.GLMForConditionalGeneration",
  "AutoModelForMultipleChoice": "modeling_glm.GLMForMultipleChoice",
  "AutoModelForSequenceClassification": "modeling_glm.GLMForSequenceClassification"
  },
...
```

So my bad, you just need to add `AutoModelForMaskedLM` !

This is a causal lm, is it ok to add it to masked lm?

Ah sorry for Causal LM it should be `AutoModelForSeq2SeqLM`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.