noamaanMulla-03/transformers-issues huggingface dataset:
Protobuf 4 support
Author: RobinKa
Pull request: no

### Feature request

Currently transformers requires protobuf 3 or lower

https://github.com/huggingface/transformers/blob/a8eb4f79f946c5785f0e91b356ce328248916a05/setup.py#L141

Support for version 4 should be added.

### Motivation

Some Python packages only work with protobuf 4 so transformers is incompatible with them (for example [flytekit](https://github.com/flyteorg/flytekit) >= 1.3).

### Your contribution

-

Comments:
Last time we checked, `protobuf>=4` was blowing up sentencepiece entirely, which is a dependency we really need in Transformers. I don't know if that has been fixed since then, maybe @ydshieh could check when he has some time?

Running T5 tokenization tests gets a lot of failure (T5 tokenizer use `sentencepiece` ), if I use `protobuf==4.22.0`

Also, I see the following conflict when I installed latest `protobuf`. 
```bash
tensorflow 2.11.0 requires protobuf<3.20,>=3.9.2, but you have protobuf 4.22.0 which is incompatible.
tensorboardx 2.5.1 requires protobuf<=3.20.1,>=3.8.0, but you have protobuf 4.22.0 which is incompatible.
tensorboard 2.11.1 requires protobuf<4,>=3.9.2, but you have protobuf 4.22.0 which is incompatible.
ray 2.0.0 requires protobuf<4.0.0,>=3.15.3, but you have protobuf 4.22.0 which is incompatible.
onnx 1.12.0 requires protobuf<=3.20.1,>=3.12.2, but you have protobuf 4.22.0 which is incompatible.
```
If `tensorflow` is installed in this case, even pytorch tests will fail, as there is
```bash
  File "/home/huggingface/transformers-hf-gcp/src/transformers/trainer_utils.py", line 47, in <module>
    import tensorflow as tf
```




So it looks like lots of libraries in our soft dependencies do not support protobuf 4 yet. We won't be able to offer support either until they do :-)
AdamW implementation
Author: StrangeTcy
Pull request: no

### Feature request

I'm getting the warning from optimization (https://github.com/huggingface/transformers/blob/main/src/transformers/optimization.py, lines 391 and on): 
```
"This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch"
                " implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this"
                " warning"
```
How worried should I really be? Are there plans to use the torch AdamW version and eventually discarding your own implementation?

### Motivation

Presumably the torch.optim.AdamW implementation is better and using it would make the whole library a bit leaner

### Your contribution

Not sure

Comments:
Hi @StrangeTcy, thanks for raising this issue! 

Please don't be worried :) The warning is there so that there isn't any unexpected changes for users when `AdamW` is eventually removed from the library and is part of the deprecation cycle. We advise that the torch implementation is used instead of the one in the transformers library, and making this switch now in the relevant places in your code will ensure that nothing breaks when the time comes. Until then, the `AdamW` class will remain in transformers. 

One thing this warning is missing is specific information about when i.e. which version, this will happen and should be added! 

Great, thanks. Looking forward to the next versions
Italian translation perf_infer_cpu
Author: nickprock
Pull request: yes

## What does this PR do?

Italian translation of doc related to the preprocessing of :hugs: Transformers.
* updated _toctree.yml
* added perf_infer_cpu.mdx

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

See issue: [[#17459](https://www.linkedin.com/feed/hashtag/?keywords=%2317459)](https://github.com/huggingface/transformers/issues/17459)


@sgugger, @stevhliu, @MKhalusova and @omarespejel 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Trainer] Add optional communication backends for torch.distributed when using GPU
Author: heya5
Pull request: yes

# What does this PR do?
Add optional backends for `torch.distributed` when using GPU.
I want to use other communication backends according the [pytorch_distribution_tutorial](https://pytorch.org/tutorials/intermediate/dist_tuto.html#communication-backends), but I found Trainer only uses nccl when `self.no_cuda is False`
.
Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
- trainer: @sgugger



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Cannot import name 'BioGptModel' from 'transformers'
Author: sky1ove
Pull request: no

### System Info

https://huggingface.co/docs/transformers/model_doc/biogpt

Cannot import name 'BioGptModel' from 'transformers'

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import BioGptModel, BioGptConfig

### Expected behavior

..

Comments:
Please follow the issue template. You likely need to upgrade your Transformers version (`pip install transformers --upgrade`) but we can't know without more information.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Proper map location for optimizer load
Author: sgugger
Pull request: yes

# What does this PR do?

I have been thinking more about #22159 and now remember why it might be better to load the optimizer state on the device directly: in multi-GPU training, the optimizer state is load in each process, so that would load it num_processes times on the CPU and risk a CPU RAM OOM.

Therefore, this adjusts #22159 to load the optimizer state:
- on CPU when there is only one process
- on each device directly when there are multiple.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
support for pytorch-directml
Author: kotoroshinoto
Pull request: no

# 🚀 Feature request

I would like to be able to work with the available models and trainer classes when using pytorch-directml under WSL

## Motivation

importing relevant huggingface packages results in a message that pytorch could not be detected and as such the models would not be available. there is an importable torch module though, provided by the pytorch-directml package.

## Your contribution

I'm not sure how many changes would be required to make this work. If the features of the fork of pytorch are close to the newest release, it wouldn't be much, but I do know that the directml implementation for tensorflow is quite out of date. (using 1.15 instead of 2 or greater)


Comments:
Hello! If you can provide a reproducible example, we're happy to take a look!

It's probably a matter of adding `pytorch-directml` to this part of the `file_utils.py` file:

https://github.com/huggingface/transformers/blob/558f8543ba3860c736a7a9a4176ac20f23f9d5a0/src/transformers/file_utils.py#L67-L77

Like it is done here for many TensorFlow flavors:

https://github.com/huggingface/transformers/blob/558f8543ba3860c736a7a9a4176ac20f23f9d5a0/src/transformers/file_utils.py#L80-L112

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

`pytorch-directml` replces `torch`
But `transformers` dont recognice/accept it  

https://pypi.org/project/pytorch-directml/
https://docs.microsoft.com/en-us/windows/ai/directml/dml-intro

is this already added? i would like to be able to run transformers with `pytorch-directml`
Fix doc links
Author: amyeroberts
Pull request: yes

# What does this PR do?

Resolves issue with some dead links in the documentation resulting from relative paths. 
Equivalent links were searched for in the translated docs but were not found. Hence only changes in files in `docs/source/en`

Fixes # 21596


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
torch.compile() and FSDP/DDP wrappers are called in the wrong order.
Author: ani300
Pull request: no

### System Info

transformers main branch

### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

When training/fine-tuning a model, activate torch.compile() and FSDP with `torch_compile=True` and `fsdp="full_shard auto_wrap"` as training arguments.

The model is compiled before the FSDP wrapping, preventing optimizations on the backwards passes. According to the PyTorch docs, both DDP and FSDP wrappers have special optimizations that run with torch.compile() to ensure model training doesn't end up slower instead of faster (see [here](https://dev-discuss.pytorch.org/t/torchdynamo-update-11-making-fsdp-and-dynamo-work-together/1037)).

### Expected behavior

Therefore, the model would need to be torch.compile()'d after being wrapped in either FSDP or DDP. Right now, in `src/transformers/trainer.py` that is not the case, with compile() being the first call in `_wrap_model()`. Before making a PR with the change, I figured I'd make this bug report to ensure nothing prevents that change from happening.

Comments:
Note that we haven't tested `torch.compile` with any kind of distributed training yet, so it's normal if there are issues. If you have the fix, we'd be happy to look at a PR!

Ok! I'll make the PR then, just figured I'd ask before.
Move torch.compile() wrapping after DDP/FSDP wrapping to ensure correct graph breaks during training
Author: ani300
Pull request: yes



# What does this PR do?

This is a simple PR that moves the wrapper for torch.compile() after those for DDP and FSDP, given that the order is important for those two pieces to work together during training.

Fixes #22215


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I don't have permissions to merge the PR, so I don't know what the process looks like from here

I was just waiting for the tests to complete ;-)
Update vision docstring bool masked pos
Author: amyeroberts
Pull request: yes

# What does this PR do?

Add the missing `bool_masked_pos` information in the docstring for vision models.

Fixes #21484 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Dead link in Documentation 📚
Author: danadascalescu00
Pull request: no

Dead link found on: https://huggingface.co/docs/transformers/tasks/summarization#inference ("_For more details about the different text generation strategies and parameters for controlling generation, check out the_ [Text Generation API](https://huggingface.co/docs/transformers/tasks/main_classes/text_generation)") . The URL should lead to the Text Generation documentation, which can actually be found at: [https://huggingface.co/docs/transformers/main_classes/text_generation](https://huggingface.co/docs/transformers/main_classes/text_generation).

Comments:
Hey there! Thanks for the report! Would you like to open a PR to update the corresponding file in `transformers`? https://github.com/huggingface/transformers/tree/main/docs/source/en/tasks 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @danadascalescu00 - thanks for raising this issue! I'm closing it now as it's resolved in #22274
Tokenizer class LLaMATokenizer does not exist or is not currently imported.
Author: ans92
Pull request: no

Hi, I want to run the LLaMA model. But I am facing issues on AutoTokenizer. I am running the following command.
```
tokenizer = AutoTokenizer.from_pretrained(Path(f"/root/text-generation-webui/models/{shared.model_name}/"))
```
But it is giving me the following error:
```
ValueError: Tokenizer class LLaMATokenizer does not exist or is not currently imported.
```
My transformers version is: 4.28.0.dev0
Python version: 3.10.9

Could you please help me in this regard.

Comments:
I have found the following answer that solved my issue:
https://github.com/huggingface/transformers/issues/22222#issuecomment-1477171703
More doctests
Author: ydshieh
Pull request: yes

# What does this PR do?

Add all files (tokenization / image processor / feature extractor / processor) to doctests.
Currently the list is not sorted - it might be better to add a check to sort the list in order (in `utils/check_doctest_list.py`).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Currently 46 failed tests - error message to be sent to Slack is too long and failed the report sending.
So I remove them for the list to be tested. Let's deal with them and add them back step by step, @amyeroberts 

For reference, here [the job run page](https://github.com/huggingface/transformers/actions/runs/4468541471/jobs/7849422399)

```bash
src/transformers/models/auto/tokenization_auto.py
src/transformers/models/bart/tokenization_bart.py
src/transformers/models/bart/tokenization_bart_fast.py
src/transformers/models/bertweet/tokenization_bertweet.py
src/transformers/models/blenderbot/tokenization_blenderbot.py
src/transformers/models/blenderbot/tokenization_blenderbot_fast.py
src/transformers/models/bloom/tokenization_bloom_fast.py
src/transformers/models/codegen/tokenization_codegen.py
src/transformers/models/codegen/tokenization_codegen_fast.py
src/transformers/models/deberta/tokenization_deberta.py
src/transformers/models/deberta/tokenization_deberta_fast.py
src/transformers/models/dpr/tokenization_dpr.py
src/transformers/models/dpr/tokenization_dpr_fast.py
src/transformers/models/gpt2/tokenization_gpt2.py
src/transformers/models/gpt2/tokenization_gpt2_fast.py
src/transformers/models/gptsan_japanese/tokenization_gptsan_japanese.py
src/transformers/models/gptsan_japanese/tokenization_gptsan_japanese.py
src/transformers/models/gpt_neox/tokenization_gpt_neox_fast.py
src/transformers/models/gpt_sw3/tokenization_gpt_sw3.py
src/transformers/models/led/tokenization_led.py
src/transformers/models/led/tokenization_led_fast.py
src/transformers/models/longformer/tokenization_longformer.py
src/transformers/models/longformer/tokenization_longformer_fast.py
src/transformers/models/luke/tokenization_luke.py
src/transformers/models/m2m_100/tokenization_m2m_100.py
src/transformers/models/marian/tokenization_marian.py
src/transformers/models/mvp/tokenization_mvp.py
src/transformers/models/mvp/tokenization_mvp_fast.py
src/transformers/models/roberta/tokenization_roberta.py
src/transformers/models/roberta/tokenization_roberta_fast.py
src/transformers/models/roformer/tokenization_roformer.py
src/transformers/models/roformer/tokenization_roformer_fast.py
src/transformers/models/transfo_xl/tokenization_transfo_xl.py
src/transformers/models/transfo_xl/tokenization_transfo_xl.py
src/transformers/models/auto/image_processing_auto.py
src/transformers/models/auto/feature_extraction_auto.py
src/transformers/models/markuplm/feature_extraction_markuplm.py
src/transformers/models/auto/processing_auto.py

```
Translate perf_infer_gpu one
Author: davidegazze
Pull request: yes

See issue [https://github.com/https://github.com/huggingface/transformers/issues/17459]
Add Italian translation of perf_infer_gpu_one.mdx and update _toctree.yml.
It's my first pull request, so I hope it's ok

The GitHub related issue is [here](https://github.com/huggingface/transformers/issues/22294)

@omarespejel
@sgugger

Comments:
Native support of ChatGLM-6b
Author: xianbaoqian
Pull request: no
Labels: name: New model

name: Feature request

### Feature request

Support https://huggingface.co/THUDM/chatglm-6b (and its int4 variants) in the Transformers library instead of relying on remote code execution.


### Motivation


This model performs really well (despite being a small model compared to large ones) and got a LOT of attention recently. It might be the SD moment for LLM IMO as it runs perfectly on consumer GPUs.

It would be great if Transformers can have native support for this model, instead of relying on remote code execution. A native integration will also make it much easier to use the model on Inference API / Endpoints. 

### Your contribution

cc @sgugger @osanseviero 

Comments:
Transformers does have native support for it even if it's not in the lib itself ;-) I see this as a chance to better support models with code on the Hub since that is the way the author chose, and since it will be more and more the norm as we cannot have the library grow exponentially.

Of course, if the authors prefer to integrate the model in the library directly, we would be happy to look at the PR and help them merge it. We can also revisit if the issue gets a lot of traction and integrate it ourselves directly.

I echo what Sylvain is saying above.

Additionally, for readers, if you would like this model to be integrated within the library nonetheless for it to be constantly tested and up-to-date with our API, please upvote the original post or add a comment mentioning so in this issue as this will help us identify models that should be more actively tested.

Thanks!

Thanks for all great inputs! Let's see how much demand we gathered for this one. 

Just for your information ChatGLM-6b is the No. 1 model on the trending page now.

<img width="288" alt="image" src="https://user-images.githubusercontent.com/38108242/226628729-b4cf69e8-8fe1-45bc-b03f-ebceb2bfce2c.png">



> This model performs really well (despite being a small model compared to large ones) and got a LOT of attention recently. It might be the SD moment for LLM IMO as it runs perfectly on consumer GPUs.

It does seem quite good but for it to be the true SD moment I think the license would have to allow commercial use, which it doesn't.
Add translation perf_infer_gpu_one for it
Author: davidegazze
Pull request: yes

[davidegazze](https://github.com/davidegazze)
davidegazze commented [52 minutes ago](https://github.com/huggingface/transformers/pull/22295#issue-1634162644) • 
See issue [https://github.com/https://github.com/https://github.com/huggingface/transformers/issues/17459]
Add Italian translation of perf_infer_gpu_one.mdx and update _toctree.yml.
It's my first pull request, so I hope it's ok

The GitHub-related issue is https://github.com/huggingface/transformers/issues/22294

@omarespejel
@sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @nickprock The PR is good to merge with your seal of approval :) 

@amyeroberts LGTM
Thanks @davidegazze