noamaanMulla-03/transformers-issues huggingface dataset:
Remove `Roberta` Interdependency from `tokenization_luke`
Author: OtherHorizon
Pull request: yes

# What does this PR do?

Related to #19303 
Removes `Roberta` dependency from `tokenization_luke.py`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger


Comments:
> Thanks a lot for working on this one! Looking good for the properties added but the main init should retain the specificities of the Luke tokenizer, otherwise it will break it :-)

Hi,thanks for the super fast review. Will fix asap.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19349). All of your documentation changes will be reflected on that endpoint.

> I'm sorry if I was unclear in my previous comments. You still need to add the init of Roberta inside the init of Luke. It's just that you should not remove the extra code Luke adds in its init :-)
> 
> Also be careful with the docstring changes, they should be reverted.

working on it

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
M2M100 training does not improve model's performance
Author: joao-alves97
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.23.0.dev0
- Platform: Linux-5.4.0-1072-aws-x86_64-with-glibc2.27
- Python version: 3.8.13
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.12.1+cu113 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help?

@patil-suraj

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hello,
I’m trying to fine-tune M2M100 using the  example script run_translation.py and it seems that the model is not improving.
I am using the following command:

```
deepspeed examples/pytorch/translation/run_translation.py \
        --deepspeed tests/deepspeed/ds_config_zero3.json \
        --model_name_or_path facebook/m2m100_418M \
        --per_device_train_batch_size 8 \
        --per_device_eval_batch_size 8 \
        --output_dir output_dir --overwrite_output_dir \
        --fp16 \
        --do_train --do_eval --do_predict \
        --max_train_samples 500 --max_eval_samples 50 --max_predict_samples 50 \
        --num_train_epochs 0.001 \
        --dataset_name wmt16 --dataset_config "ro-en" \
        --source_lang en --target_lang ro \
        --predict_with_generate --forced_bos_token ro
```
    
Just to give you an example, if I train for 1 epoch I can get 20 BLEU points in the test set, but if I train for 3 epochs I get around 10 BLEU points.
Am I doing anything wrong? Does M2M100 requires any specific hyperparameter/hyperparameter configuration?
Thanks

### Expected behavior

I was expecting the model performance to improve for this specific language-pair (en-ro)

Comments:
@patil-suraj could you help me with this problem?

Hi @joao-alves97, you should really ask those questions on the [forums](https://discuss.huggingface.co/) where the whole community will be able to help. We keep GitHub issues for bugs (with a reproducer) or feature request only :-)

I already did it @sgugger and no one answered me: https://discuss.huggingface.co/t/m2m100-training-does-not-improve-model-performance/23787 . And it seems to be a bug in my opinion. It is not normal that the loss continues decreasing but the performance does not improve at all.  Something must be wrong in the M2M100 implementation. I did an exhaustive hyperparameter search and it did not solve the issue.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
remaining pytorch type hints
Author: IMvision12
Pull request: yes

# What does this PR do?

Type hints

@Rocketknight1

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20217). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20217). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20217). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20217). All of your documentation changes will be reflected on that endpoint.

@Rocketknight1 There is only 1 left: EsmForProteinFolding. when I tested test_modeling_esmfold.py before doing any changes to the file some tests where initially getting failed.

![Capture](https://user-images.githubusercontent.com/88665786/202189778-224a16b0-4926-4adc-9fef-c9b87ca1b161.PNG)


@IMvision12 Those tests shouldn't be failing anymore, but don't worry - I'm happy to accept all the other changes!
Fix no trainer summarization script test failure
Author: muellerzr
Pull request: yes
Labels: name: Examples

# What does this PR do?

This PR solves the test failure in the nightly CI for the summarization no trainer script. Also potentially https://github.com/huggingface/transformers/issues/18189 too, just waiting for verification first otherwise a followup PR will be made

`gather_for_metrics` was being called twice instead of once, along with a syntax error, which is not good! :) 

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20286). All of your documentation changes will be reflected on that endpoint.
[DOCTEST] Fix the documentation of RoCBert
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the documetnation test


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20142). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20142). All of your documentation changes will be reflected on that endpoint.

Sorry for the late fix, tests are passing locally. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20142). All of your documentation changes will be reflected on that endpoint.
Saving a 8-bit T5 does not work.
Author: peregilk
Pull request: no

### System Info

Latest.

### Who can help?

@patrickvonplaten 

I am able to load a T5 model in 8-bit-format using this command:
`model = T5ForConditionalGeneration.from_pretrained(".",load_in_8bit=True, device_map="auto")`

The model works fine. Saving the model using ".save_pretrained()" also seems to work, and the file on the disk is a lot smaller. I am able to load the save model again without errors, but then the model no longer works.

Here are two sample models:
north/fine_North_large
north/fine_North_large_8bit


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

See description above

### Expected behavior

The model should not change when being saved.

Comments:
Those are all arguments for inference specifically. `save_pretrained` does not work if you use `device_map="auto"` if you have weights offloaded on the CPU (not sure if it's the case or not here) and it's not been tested with `load_in_8bit` either.

We will probably fix this later down the road, but for now you should not save on disk the model this way (you can find the model in your cache if you need its weights anyway)

Thanks for the feedback, @sgugger.

So, just to be sure that I understand this: There is not really any way of easily distributing only the 8-bit model. The correct way of doing it will be to distribute the 32-bit version, and then ask people to use **load_in_8bit=True, device_map="auto"** ?

What about loading the 8-bit-version in the HuggingFace widgets. Is that even possible?

There is no way to save and reload an 8-nit model anyway (correct me if I'm wrong @younesbelkada ) so you need to use `load_in_8bit=True, device_map="auto"` indeed.

Yes this is correct, you can also save the model in fp16 to save memory and load it back in int8, but currently saving and loading the model in 8-bit is not supported
[bnb] Let's warn users when saving 8-bit models
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR warns users that try to save 8-bit loaded models

closes #20247
closes #19480

cc @sgugger 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20282). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20282). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20282). All of your documentation changes will be reflected on that endpoint.

Ah ah made a typo in one of my suggestions, sorry. `getattr` has less `t`s ;-)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20282). All of your documentation changes will be reflected on that endpoint.

Thanks for implementing this. This fix would absolutely have prevented the confusion explained in: [#20247](https://github.com/huggingface/transformers/issues/20247).

Thank you very much @peregilk ! 🙏
Image transforms functionality used instead
Author: amyeroberts
Pull request: yes

# What does this PR do?

* Removes reimplementations of `center_to_corners` format
* Removes `# Copied from` statements and imports directly instead


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20278). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20278). All of your documentation changes will be reflected on that endpoint.
TF: add test for `PushToHubCallback`
Author: gante
Pull request: yes

# What does this PR do?

Adds a test to TF's `PushToHubCallback`

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20231). All of your documentation changes will be reflected on that endpoint.

Could you ensure it works with `huggingface_hub` version `v0.11.0rc1`? I think there was an issue with this callback.

cc @Wauplin 

To be more precise, it should already work with `v0.11.0rc1` (was not the case with `v0.11.0rc0`) BUT a warning message should be triggered "Creating a repository through 'clone_from' is deprecated". To prevent this warning, `PushToHubCallback` should create the repo before cloning it. Something like:
```py
create_repo(repo_id, exists_ok=True)

repo = Repository(clone_from=repo_id)
```

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20231). All of your documentation changes will be reflected on that endpoint.

All tests are now passing with `HUGGINGFACE_CO_STAGING=1 py.test tests/test_modeling_tf_common.py::TFModelPushToHubTester -vv`, tested against version `0.10.1` of the hub.

@Wauplin I've factored in the `create_repo()` before cloning the repo 👍 

@Rocketknight1 it was indeed failing because of the lack of `compile()`. After the fix, I ran against a problem where the last epoch was not being stored (because of the process kill). Let me know if you agree with the changes in `PushToHubCallback`

Nice, thanks for making the change @gante !
Generate: general TF XLA constrastive search are now slow tests
Author: gante
Pull request: yes

# What does this PR do?

TF's XLA contrastive search tests were time-consuming because of the conversion to XLA, so this PR moves them to more powerful slow tests.

Making these tests faster would imply creating smaller model configs for each model, which seems like overkill.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Make size_dict conversion logs clearer
Author: amyeroberts
Pull request: yes

# What does this PR do?

* Tidies up logic for converting `size` parameter to the expected dictionary format for image processors.
* Adds `param_name` as a flag so logs reflect the variable being updated e.g. `crop_size` versus `size`

Address part of #20185  - trying to make the logs clearer. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
@sgugger @patrickvonplaten  The quickest and easiest remedy I had was adding a `param_name` flag for the `get_size_dict` function. This way, if an image processor has both `size` and `crop_size` variables being updated, the logs reflect the parameter being changed. However, it's a bit of a dirty trick. LMK if you have an alternative suggestion. 



The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20205). All of your documentation changes will be reflected on that endpoint.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20205). All of your documentation changes will be reflected on that endpoint.

Thanks!
Add Galactica model
Author: mrm8488
Pull request: no
Labels: name: New model

### Model description

Galactica is a large language model for science.
Can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins.

Web: galactica.org
Code: https://github.com/paperswithcode/galai

PS: It seems to use OPT architecture so maybe we can reuse the code used for adding these model. I would like to add this model :)

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
If the architecture is the same, it's just a matter of converting the checkpoints to the HF format and uploading them, no need to add a new model in the library :-)

Hi, Sylvain!

Yes, It is the same (Even I have run the forward pass with their wrapper and the HF `OPTForCausalLM` and the output is the same. I think the ckpts are already converted (They have a config file that makes them work with HF).

2 questions:

1) The tokenizer is read from a single file `tokenizer.json` with the `Tokenizer.from_file()` method. How do I split it into separated files? (config, merges, vocab, etc)
2) Do I upload the models with my personal account? Like this: https://huggingface.co/mrm8488/galactica-125m ?

Thanks!

Not working correctly right now.

@sgugger Can you guide us better?

@mrm8488 would probably be best to include under the `facebook` org, some folks at HF like @patrickvonplaten should be able to upload them there

@patrickvonplaten We need your help on this one.

Hey guys, stoked for the integration - just one model specific thing, if user enters prompt like:

"[START_AMINO]MVATE[END_AMINO]" -> need to tokenize so the bit inside the special tokens is character-based tokenized -> i.e M, V, A, T, E. we have the logic in https://github.com/paperswithcode/galai/blob/main/galai/model.py, see "escape_custom_split_sequence".

Any update?

> Any update?

It is WIP! Thanks! :)

Model is fully added see: https://huggingface.co/models?other=galactica - thanks a lot @mrm8488 for you work here! Closing this :-)
Different behaviour of AutoTokenizer and T5Tokenizer
Author: pietrolesci
Pull request: no

### System Info

- `transformers` version: 4.22.2
- Platform: Linux-5.4.209-129.367.amzn2int.x86_64-x86_64-with-glibc2.26
- Python version: 3.9.13
- Huggingface_hub version: 0.10.0
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: NO
- Using distributed or parallel set-up in script?: NO

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The `T5Tokenizer` prepends a whitespace before the eos token when a new eos token is provided, while AutoTokenizer maintains the usual behaviour.

```python
from transformers import AutoTokenizer, T5Tokenizer

text = ["My name is Pietro", "I love pizza"]

tok = T5Tokenizer.from_pretrained("t5-small", bos_token="[bos]", eos_token="[eos]", sep_token="[sep]")
auto_tok = AutoTokenizer.from_pretrained("t5-small", bos_token="[bos]", eos_token="[eos]", sep_token="[sep]")
print(tok.batch_decode(tok(text)["input_ids"]))
print(auto_tok.batch_decode(tok(text)["input_ids"]))

#> ['My name is Pietro [eos]', 'I love pizza [eos]']
#> ['My name is Pietro[eos]', 'I love pizza[eos]']


tok = T5Tokenizer.from_pretrained("t5-small")
auto_tok = AutoTokenizer.from_pretrained("t5-small")
print(tok.batch_decode(tok(text)["input_ids"]))
print(auto_tok.batch_decode(tok(text)["input_ids"]))

#> ['My name is Pietro
', 'I love pizza
']
#> ['My name is Pietro
', 'I love pizza
']
```

### Expected behavior

The two tokenizer classes should be equivalent

Comments:
When you're using `T5Tokenizer`, you're loading the slow version. When using `AutoTokenizer`, you're loading the fast version by default. 

Unfortunately, due to a difference in implementation, the fast and slow tokenizer can have some differences.

Pinging @SaulLu and @ArthurZucker for knowledge

Yes, noticed the same thing with GPT2, especially with OOV that are automatically converted to blank with the fast version! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[bnb] Simplifies slow test
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR simplifies a slow test for `bnb`. In fact, you can easily retrieve the devices of the model using `set(model.hf_device_map.values())` 

cc @sgugger 
Thanks! 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
position_ids cannot be specified for GPTNeoXForCausalLM
Author: nkandpa2
Pull request: no

### Feature request

The GPTNeoXForCausalLM class' forward method does not support passing `position_ids`. This model class uses rotary positional encodings so `position_ids` are needed for correct forward passes on left padded sequences. 

### Motivation

The GPTNeoXModel class uses rotary positional encodings so `position_ids` are needed for correct forward passes on left padded sequences. Additionally, the documentation (https://huggingface.co/docs/transformers/model_doc/gpt_neox#transformers.GPTNeoXForCausalLM) list `position_ids` as an argument for `forward` but this is not consistent with the implementation. 

### Your contribution

I've taken a look through the code and it's not immediately obvious how to implement this. I'd be happy to contribute but may need some pointers.

Comments:
Hi @nkandpa2 👋 

We are aware of this problem (see also https://github.com/huggingface/transformers/issues/17283), and we are working on a fix (https://github.com/huggingface/transformers/pull/18048)

As you wrote, it is not a trivial change :D 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.