add model resources for CPMAnt (new)
Author: pioliverse
Pull request: yes

# What does this PR do?
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->
Since the previous submission(#20711 ) had problems here and there, we have now resubmitted a new one.
## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger and @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
> Thanks very much @pioliverse for iterating! I left a couple of comments, I think that some refactoring needs to be considered, after that we should be close to merge this! My main comments are:
> 
> * I think that you can wrap `CPMAntEmbedding` around a `nn.Embedding` layer even though scaling is needed. You can scale down after each call to the embedding module and make sure the input is scaled down before the `projection` call.
> * Make sure to inherit `CPMAntForCausalLM` from `CPMAntPreTrainedModel`, also make sure to follow the convention / good practices by checking what is done in OPT for instance: https://github.com/huggingface/transformers/blob/1543cee7c8c95ef47f832b1f37625ba2923c4994/src/transformers/models/opt/modeling_opt.py#L808
>    - this includes defining correctly a `lm_head` module, functions such as `get_input_embeddings`, `set_input_embeddings`, etc.
> * A lot of arguments from module's init seems to be unused, e.g. `init_std`. Try also to take the `config` object as a single argument from the init whenever possible (e.g. `CPMAntEncoder`)
> * Please make sure to follow the correct styling for docstrings (check my comments about that below)
> * If you have to initialize some weights with a specific distribution, try to initialize all the submodules weights inside `_init_weights` function from `CPMAntPreTrainedModel`
> * It's unclear to me why `forward` function is not defined in `CPMAntForCausalLM`
> * The code can be optimized here and there, I left some comments below on how you can achieve that
> * Please do not raise `RuntimeErrors` outside `if torch_is_available()`, otherwise `flax` & `tf` tests will fail
> Again thanks a lot for your efforts!

@younesbelkada Thanks for your patience in reviewing, I followed OPT convention and made the following changes:
> * `CPMAntEmbedding` and `CPMAntLinear` has been replaced by `nn.Embedding` and `nn.Linear` respectively.
> * `CPMAntForCausalLM` has been inherited from `CPMAntPreTrainedModel`, and `lm_head` and some functions have been added.
> * Useless initial arguments have been removed.
> * `forward` has been defined in `CPMAntForCausalLM`

@younesbelkada Thanks again for your patience in reviewing.

_The documentation is not available anymore as the PR was closed or merged._

_The documentation is not available anymore as the PR was closed or merged._

> Thanks so much for your patience ! Looks pretty clean thank you! We should be close merging this once most of the comments are addressed. My comments being:
> 
> ### Docstring and comments:
> * please harmonize the function docstrings to match the convention `transformers` model follow
> * please make sure to clean up some comments
> * Also would be nice to add a small explanation on the code on why `generate` needs to be overriden
> 
> ### `dtype`:
> * I don't think the argument `dtype` is needed. The dtype of the whole model is managed by the kwarg `torch_dtype` so you can load your model using `model = xxxForCausalLM.from_pretrained(xxx, torch_dtype=torch.float16)` or `torch_dtype="auto"`  (if the weights are pushed in fp16) and the model will be loaded in the desired precision.
> 
> ### tests
> * I think that a test is failing, please double check that
> 
> ### general comments
> * For classes that are public (i.e. that are ported in `__init__.py`, basically`CPMAntModel` &. `CPMAntForCausalLM` it is preferable to adopt this logic: https://github.com/huggingface/transformers/blob/main/src/transformers/models/gpt_neox_japanese/modeling_gpt_neox_japanese.py#L693-L703 --> outputa tuple if not `return_dict` otherwise return a dataclass. Please check other modeling files as reference
> * you can wrap `attention_mask` creation process inside class methods, e.g. `_prepare_attention_mask(`
> 
> Thanks!



Hi @younesbelkada, we have made some changes as follows:
1. add some docstrings.
2. modified `forward` following the style of transformers.
3. rewrote some functions to adapt the `generate` function
>* in `modeling_cpmant.py`, we rewrote some functions like `prepare_inputs_for_generation`, `_expand_inputs_for_generation`
>* in `tokenization_cpmant.py`, rewrote some functions like `prepare_for_model`, `_pad`, `_encode_plus`, `_batch_encode_plus`
4. cleaned some comments.


I am a bit surprised that when I use `make style`, some other files are also reformatted, which causes `check_code_quality` to fail.

Hi @pioliverse 
You need to rebase with `main` branch as the styling has been updated for most of the files in `transformers` , and update your black version as follows:
```
pip install --upgrade -e .["quality"]
```
Then `make style` or `make fixup`

> Hi @pioliverse You need to rebase with `main` branch as the styling has been updated for most of the files in `transformers` , and update your black version as follows:
> 
> ```
> pip install --upgrade -e .["quality"]
> ```
> 
> Then `make style` or `make fixup`

Thanks @younesbelkada , this has been solved.

> Thanks a lot for addressing most of the comments of the previous review! And thank you for your huge work on refactoring the modeling script I left some comments, mostly nits that can be solved easily. Note that for arguments such as `use_cache` etc, we prefer to pass them through the forward pass rather than setting them as a class attribute. Also, please consider passing a `CPMAntConfig` for the classes that have several attributes such as `CPMAntEncoder` Make sure also to correctly pass the required keyword arguments such as `past_key_values`, `output_attentions` etc, that are crucial for caching mechanism. You can check how this is done in OPT for example Finally, the naming convention in `transformers` has changed a bit, we prefer to name models with a single capital letter (i.e. here `CPMAnt -> Cpmant`) Again thanks for your efforts on this! Once the comments being solved, we should be very close merging this!



Thanks for your review @younesbelkada , we have modified some code.
>* We pass the `use_cache` in `forward` function from a class attribute.
>* We simplify the code for the class attribute assignment and replace it with `CPMAntconfig`.
>* We added `past_key_values` and `output_attentions` in `forward` of CPMAntModel.
>* I kind of wonder if all files that contain the name `CPMAnt` should be changed to `Cpmant`?

Hi @younesbelkada , I am a member of OpenBMB, and I will help @pioliverse finish this PR.

All the issues mentioned above have been resolved. Please kindly have a look.

For the unit tests, I rebase `pioliverse:cpmantmodel` with `huggingface:main`, but it cannot pass the test. It seems some other models cause the failure?

for instance, in tests_onnx I met the error:
`
ERROR tests/models/altclip/test_modeling_altclip.py
============ 72 passed, 551 skipped, 29 warnings, 1 error in 28.26s ============
`
How can I avoid such error?

Hi @gongbaitao 
Thanks for jumping in! And sorry for the delay
Rebasing with `main` should be probably solve this issue, will look into the PR asap, let me know once you think this is ready for review!

Hi @younesbelkada, thanks for your reply and advice!
All the problems in unit test have beed solved. In the latest commit, we largely refactor the code to make it clear and simple. Hope you can give a review soon! 

Thanks a lot for the heads-up @pioliverse @gongbaitao !
Quickly looking at the README_ja file it seems that a some unnecessarly changes were made, I suggest you merge this branch with the upstream `transformers` `main` branch:
```
git remote add upstream https://github.com/huggingface/transformers.git
git fetch upstream
git merge upstream/main
git push
```

I'll have a closer look on the other files asap! 

> Thanks a lot for the heads-up @pioliverse @gongbaitao ! Quickly looking at the README_ja file it seems that a some unnecessarly changes were made, I suggest you merge this branch with the upstream `transformers` `main` branch:
> 
> ```
> git remote add upstream https://github.com/huggingface/transformers.git
> git fetch upstream
> git merge upstream/main
> git push
> ```
> 
> I'll have a closer look on the other files asap!

Thanks for the tips @younesbelkada!
 I have merged with `huggingface main` branch and update the README_ja, and it's up-to-date with the `main` now. We hope to merge this PR in this week, please kindly have a look. Thanks!

> Regarding slow integration tests, we might need to totally skip them as the weights are 40GB in fp32 (and 20GB in fp16), I think our daily CI runners have 16GB GPU VRAM, so friendly pinging here @ydshieh to see what could be the alternative.

Yeah, just skip it/them.

Hi @gongbaitao 
Great work on refactoring the code and making the CI tests pass! 🎉 
Let me know once this is ready for review!

Hi @younesbelkada, thanks for your comment yesterday! This helps me a lot to find hidden errors in unit test part. I have fixed these bugs and make the new commit.
But I don't know why ci/cicleci: test_tf always failed with `tests/models/opt/test_modeling_tf_opt.py::TFOPTModelTest::test_pipeline_text_generation` time out, even I have tried 3 times.
Is there anything I missed?

Hello @gongbaitao 
Don't worry I think this is fine. If you give me the green light, I can review the PR now

> Hello @gongbaitao Don't worry I think this is fine. If you give me the green light, I can review the PR now

Yeah it's solved by luck i guess.
I think It's ready for review, thanks for your help again! @younesbelkada 

Hi @younesbelkada, thanks for your meaningful comments!

1. The link for checkpoint, and some other comments problems, have been corrected.

2. As for the name, because till now CPMAnt has just one Chinese case, I guess it's no need to call it like CPMAntChinese. 

3. Besides, I have made some local tests on tokenizers, for example, the `CPMAntTokenizationTest.test_pre_tokenization()`. But some methods in `TokenizerTesterMixin` use different logic to load vocab as a `dict`, while CPMAnt has it's own `load_vocab`. Refactor is not that convenient and necessary i think, so I just set it as `custom`.  I can make a change if there's any better solutions.

It's ok for the new review : )

Hi @younesbelkada . I have fixed  the problems mentioned in comments. I think it's ready for new review:)
Thanks for your detailed review and comments!

@younesbelkada @sgugger Thanks for the valued comments!
According to the new comments, I have dropped some redundant codes, and rename the model class in a camel-cased way
: )

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @sgugger @younesbelkada , sorry for the delay!
In the last few weeks, I have fixed the problems mentioned above and refactored the CPMAnt tokenizer. Please kindly have a look again, thanks for your help!

Thanks for your quick review! @sgugger 
It seems this problem https://github.com/huggingface/transformers/pull/20906#discussion_r1161687724 is because the changed file didn't show all commits. Maybe check this page https://github.com/huggingface/transformers/pull/20906/files will be helpful:)
As the https://github.com/huggingface/transformers/pull/20906#discussion_r1161688147, it cannot pass the code quality check, so shall I keep it unchanged?

@sgugger Thanks for your meaningful comments!
Sorry I forget to drop the trailing comma in styling issue. Now I have fixed the trailing comma problem and add `tooslow` decorator. Please kindly have a review:)