noamaanMulla-03/transformers-issues huggingface dataset:
[tests] switch to torchrun
Author: stas00
Pull request: yes

This PR fixes the following errors in nightly CI tests 
```
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_apex
1422
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_run_seq2seq_ddp
1423
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_trainer_log_level_replica_0_base
1424
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_trainer_log_level_replica_1_low
1425
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_trainer_log_level_replica_2_high
1426
FAILED tests/extended/test_trainer_ext.py::TestTrainerExt::test_trainer_log_level_replica_3_mixed
```


by switching from deprecated `distributed.launch` to `distributed.run`
```
:   File "/workspace/transformers/examples/pytorch/translation/run_translation.py", line 664, in <module>
384
stderr:     main()
385
stderr:   File "/workspace/transformers/examples/pytorch/translation/run_translation.py", line 262, in main
386
stderr:     model_args, data_args, training_args = parser.parse_args_into_dataclasses()
387
stderr:   File "/workspace/transformers/src/transformers/hf_argparser.py", line 341, in parse_args_into_dataclasses
388
stderr:     raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
389
stderr: ValueError: Some specified arguments are not used by the HfArgumentParser: ['--local-rank=1']
390
stderr: /opt/conda/lib/python3.8/site-packages/torch/distributed/launch.py:181: FutureWarning: The module torch.distributed.launch is deprecated
391
stderr: and will be removed in future. Use torchrun.
392
stderr: Note that --use-env is set by default in torchrun.
393
stderr: If your script expects `--local-rank` argument to be set, please
394
stderr: change it to read from `os.environ['LOCAL_RANK']` instead. See
395
stderr: https://pytorch.org/docs/stable/distributed.html#launch-utility for
396
stderr: further instructions
```

I updated `tests/trainer/test_trainer_distributed.py` while at it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Unfortunately, conda refuses to install `libstdcxx-ng=12`, and gives a super super long report of conflict packages after 20 or more minutes of examization.

I think we can merge this PR first. And I can try if there is anyway to make to get `GLIBCXX_3.4.30` installed.
@stas00  Does this work for you?

The `GLIBCXX_3.4.30` is totally unrelated to this issue so let's deal with it separately.
Update warning levels
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR updates the warning levels at 2 places:

- use logger.info instead of logger.warning when an image processor gets loaded based on a preprocessor_config.json file.
- use `logger.warning_once` instead of warnings.warn(..., FutureWarning) for DETR and friends.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix docstrings for TF BLIP
Author: Rocketknight1
Pull request: yes

Some of the docstrings were still a bit PyTorchy, this is fixed now! (cc @ydshieh)

Comments:
@Rocketknight1 Thanks for adding tf-blip. 
PS. in case you're interested to contribute https://github.com/keras-team/keras-nlp/issues/941

@Rocketknight1  I tried to re-run the CI, but it still fails. Could you push an empty commit to trigger it maybe?

_The documentation is not available anymore as the PR was closed or merged._

@ydshieh tests are passing now! Can you approve the PR?

@Rocketknight1 When I run the doctest

```python
python3 -m pytest -v --make-reports doc_tests_gpu --doctest-modules src/transformers/models/blip/modeling_tf_blip.py::transformers.models.blip.modeling_tf_blip.TFBlipForConditionalGeneration.generate -sv --doctest-continue-on-failure --doctest-glob="*.mdx"
```
I got
```
Expected:
    two cats are laying on a couch
Got:
    two cats sleeping on a couch
```

Also, the changes are not just docstrings. After looking at the changes, I am wondering why we don't have CI failures (i.e. the usual testing).

It turns out that we do have  some failures
https://github.com/huggingface/transformers/actions/runs/4653835201/jobs/8235101265

@Rocketknight1 Do you want to verify/fix those in this same PR? (probably already fixed as CircleCI is green?)

Oh, that's very odd - I wonder why it's not visible in the CI? I'll take a look!

Thank you @Rocketknight1 . I am also confused why CircleCI is green but failed on daily CI.

@ydshieh tests should pass now! The cause was some expected values in the tests being wrong. I copied the right ones from the torch tests and now everything is passing locally, so hopefully the CI will agree.

The doctests are weird, though - I think some of them were broken in PyTorch too. Working on it!

Thanks @Rocketknight1 I will double check. But do you figure out (some of) tests pass on CircleCI but fails on daily CI - the pt<->tf equivalence tests also run on CircleCI. We should see they fail on it (as they fail on daily CI).

For the doctest, we still get 

```python
Expected:
    two cats are laying on a couch
Got:
    two cats sleeping on a couch
```
for `TFBlipForConditionalGeneration.generate`.

Regarding the modeling tests - all tests pass now 🥳 

@ydshieh should be resolved now!
Token batching
Author: bpopeters
Pull request: no

Hello,

Many frameworks support _token batching_, in which batches are constructed not so that they contain the same number of sequences, but rather so that they contain approximately the same number of tokens (so a batch could consist either of a large number of short sequences or a small number of long sequences). One motivation for this is so that memory use is roughly constant from batch to batch, which makes it easier to use a very large batch size without risking an out-of-memory error.

For example, this is the behavior when using `--max-tokens` instead of `--batch-size` in fairseq.

I found a previous issue (https://github.com/huggingface/transformers/issues/14767) where this was asked. At the time, someone claimed that the feature existed and posted a video. However, the examples presented in that video do **not** actually implement this feature. Subsequent comments pointed out that the issue remained unresolved, but they were ignored.

So my question is, does token batching already exist in transformers? If so, how can I make use of it?

Thank you for your help! i wasn't sure if I should have made this a feature request because it's not actually clear to me whether the feature has already been implemented or not.

Comments:
Hi there, what you are asking for is not supported. Note that Transformers is primarily a library of models. You can adapt the data preprocessing part of any of our existing examples to suit your needs, but we won't support every feature out of the box as it's not the goal of the library.

Hello,

Thank you for your quick reply. I'll admit I'm a bit surprised that this is considered out of scope. It is a models library, yes, but the main ways people interact with models are through training (including finetuning) and inference. In either case, inputs need to be batched. This is a very mainstream technique for doing it, especially for self-attention-based models because of the popularity of very large batches (at least when training from scratch, I'm fairly new to finetuning so perhaps the situation is different).

Thank you again for your help.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Token batching is a necessary feature for some tasks like machine translation as it is a recognized setting in the field. When you want to make sure that your experimental setup is consistent with other frameworks, you must do so.
[Doctest] Add configuration_mvp.py
Author: elabongaatuo
Pull request: yes

Adds configuration_mvp.py to utils/documentation_tests.txt

Based on https://github.com/huggingface/transformers/issues/19487

@ydshieh can you please have a look?this passes the test as well, with two warnings.thank you :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[Doctest] Add configuration_m2m_100.py
Author: elabongaatuo
Pull request: yes

Add configuration_m2m_100.py to utils/documentation_tests.txt for doctest.

Based on issue https://github.com/huggingface/transformers/issues/19487

@ydshieh can you please have a look? thanks :D

it passes the test with a few warnings.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for your assistance @ydshieh 😃
Make FlaxPreTrainedModel a Flax Module
Author: cgarciae
Pull request: yes

Moved to #22627 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22479). All of your documentation changes will be reflected on that endpoint.

Thanks for the review @sanchit-gandhi !

> => perhaps as a start we first make a PR that triggers this warning (advising users that the functinality is going to change in N months / releases time), and then have this PR as a follow-up that makes the changes?

I was planning on entirely deleting the `params` argument from `__call__` 😅. If we want to make the change a bit more gradual then maybe we could do something like this:

```diff
- if self._do_init:
+ if self.scope is None:
```
This would condition on whether the module being called inside `apply` or not.

Closing in favor of #22627

Thanks for the clarifications here @cgarciae! 🙌
pointer to transformer (big) model
Author: anirudt
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

<!-- Important information -->
Hi, needed a pointer on how to instantiate a Transformer-big from the original Vaswani et. al. paper (Attention Is All You Need). I could only find versions of Transformer-like architectures, so would be useful if this could also be added.

## Open source status

* [x] the model implementation is available: (give details): https://research.google/pubs/pub46201/
* [ ] the model weights are available: (give details)
* [ ] who are the authors: (mention them, if possible by @gh-username)


Comments:
@anirudt  Can I work on this issue?

Any leads on this?
Make vilt, switch_transformers compatible with model parallelism
Author: Xrenya
Pull request: yes

# Make vilt, switch_transformers compatible with model parallelism

Fixes https://github.com/huggingface/transformers/issues/22561#issue-1653950092


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

- PyTorch: @sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

_The documentation is not available anymore as the PR was closed or merged._

@sgugger I fixed the issue, but, unfortunately, I have closed and reopened PR to trigger the CircleCI.

Thanks a lot!
[Pix2struct] Simplify generation
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR aims to fix the warning that is currently printed out when generating text with Pix2Struct:

```
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.
```
I see that all Pix2Struct models have `config.is_encoder_decoder=False`, but as Pix2Struct is an encoder-decoder model it'd be great/more logical to have this argument set to `True` and instead overwrite `prepare_inputs_for_generation` to have a cleaner way of generating text. This also makes us get rid of the warning.

To do:

- [ ] for the moment there're still one integration test failing (`test_batched_inference_image_captioning_conditioned`):

```
AssertionError: 'An photography of the Temple Bar and a collection of other items.' != 'An photography of the Temple Bar and a few other places.'
E       - An photography of the Temple Bar and a collection of other items.
E       ?                                        ^^^^ ^^^^^^^^       ^^ -
E       + An photography of the Temple Bar and a few other places.
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._

PR is ready for review, however checkpoints on the hub will need to be updated (`is_encoder_decoder` = True) for this PR to be merged

PR is ready, models on the hub don't need to be updated since they don't have `is_encoder_decoder` set on the model config level (i.e. `Pix2StructConfig`. They have set it only in `Pix2StructTextConfig`). cc @younesbelkada
Change `torch_dtype` to str when `saved_model=True` in `save_pretrained` for TF models
Author: ydshieh
Pull request: yes

# What does this PR do?

One issue in #22731 is: the config contains `torch_type` with a torch dtype class as value. Usually, our `save_pretrained` will take care of it. But when `saved_model=True` in `save_pretrained` for TF models, it is not handled, and TF/Keras complains about it.

See the first part in [this comment](https://github.com/huggingface/transformers/issues/22731#issuecomment-1506538658)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
add onnx support for llama
Author: sam-h-bean
Pull request: yes

# What does this PR do?

Add ONNX serialization support for Llama models.

Fixes # (issue)
https://github.com/huggingface/optimum/issues/918

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@JingyaHuang @chainyo

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @michaelbenayoun
Unable to load a pretrained model 
Author: poojitharamachandra
Pull request: no

### System Info

- `transformers` version: 4.18.0
- Platform: Linux-4.18.0-425.10.1.el8_7.x86_64-x86_64-with-glibc2.28
- Python version: 3.10.4
- Huggingface_hub version: 0.2.1
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


OSError: Can't load config for '~/Unixcoder/model/changesets_model'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '~/Unixcoder/model/changesets_model' is the correct path to a directory containing a config.json file

I am trying to load a second model during the course of training the first model. 
I am able to load the model for the first time, but not for the second time(even though all the config files are present)
I am trying to load the model downloaded from https://huggingface.co/microsoft/unixcoder-base/tree/main

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

1. load the model from downloaded version of https://huggingface.co/microsoft/unixcoder-base/tree/main
2. train the model on custom dataset
3. load the similar model that is pretrained on different dataset

### Expected behavior

model is successfully loaded 

Comments:
Could you paste the code you are using?
```py
from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("microsoft/unixcoder-base")
model = AutoModel.from_pretrained("microsoft/unixcoder-base")
```
works perfectly fine.

from transformers import (WEIGHTS_NAME, AdamW, get_linear_schedule_with_warmup,
                              RobertaConfig, RobertaModel, RobertaTokenizer)   

 tokenizer = RobertaTokenizer.from_pretrained(args.model_name_or_path)
    config = RobertaConfig.from_pretrained(args.model_name_or_path)
    model = RobertaModel.from_pretrained(args.model_name_or_path) 
.............................






    model = RobertaModel.from_pretrained('./changesets_model')
    model = Model(model)
    model.to(args.device)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Bug in Seq2SeqTrainer?
Author: qmeeus
Pull request: no

### System Info

Transformers version: 4.28.0.dev0
Pytorch version: 2

### Who can help?

@sanchit-gandhi @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Finetune Whisper on multiGPU, using Seq2SeqTrainer

During generation step, I am getting an `AttributeError` ('DataParallel' object has no attribute 'generation_config') due to [this line of code](https://github.com/huggingface/transformers/blob/main/src/transformers/trainer_seq2seq.py#L280). Note that `generation_config` is a correct attribute in the model. 

I did finetune a modified version of Whisper on 2 GPUs in the past, but I did not experience the same error. I think that previously, generation was done on one GPU, even if the trainer was using data parallel. Not exactly sure if something has changed or if I did something wrong

The model is `openai/whisper-large-v2`

### Expected behavior

Either generation should not use `DataParallel` or `Seq2SeqTrainer` should access `model.model.generation_config` if the model is an instance of `DataParallel`

Happy to write a PR if necessary, let me know



Comments:
This has been fixed on main, so make sure to do a source install (or in a couple of hours you can upgrade to v4.28.0 once it's released).

Shame on me, I did not pull the latest changes

Thank you for your quick answer

No worries at all!
🌐 [i18n-KO] Translated `training.mdx` to Korean
Author: gabrielwithappy
Pull request: yes

# What does this PR do?

Translated the `training.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

 May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo

<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->
<!-- Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd -->

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Team PseudoLab, may you please review this PR?
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd
- fixed some ambiguous translations of the autoclass_tutorial doc. 
(from https://github.com/huggingface/transformers/pull/22533)
- translated training doc.

fixed review comments :-) 
Thank you your kind reivews.
BRs
@HanNayeoniee @sim-so @wonhyeongseo 

May you please review this PR?
@sgugger, @ArthurZucker, @eunseojo
`DocumentQuestionAnsweringPipeline` only for fast ⚡ tokenizers
Author: ydshieh
Pull request: yes

# What does this PR do?

So let's make this more clear and explicit.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22745). All of your documentation changes will be reflected on that endpoint.
Remove `DS_BUILD_AIO=1`
Author: ydshieh
Pull request: yes

# What does this PR do?

In order to make 
```
PASSED tests/deepspeed/test_deepspeed.py::TrainerIntegrationDeepSpeed::test_stage3_nvme_offload
```
Note it pass, not skipped. I am not sure if this is normal however, but you can see the results in [this job run](https://github.com/huggingface/transformers/actions/runs/4686799372)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

funny, it means that it built it at runtime and it worked. So it finds the right library at run time.

So actually if we wanted to skip the test we would need to removed apt install libaio-dev.

But this is a better outcome, so let's keep it the way you proposed.
fix url post pt-2.0 release
Author: stas00
Pull request: no

change:
https://pytorch.org/docs/2.0/generated/torch.compile.html?highlight=torch+compile#torch.compile
to:
https://pytorch.org/docs/stable/generated/torch.compile.html?highlight=torch+compile#torch.compile
once the latter doc appears post pt-2.0 release

for the trainer code here:
https://github.com/huggingface/transformers/pull/22140

Actually it looks like all the good stuff is at https://pytorch.org/docs/master/dynamo/index.html - but again be wary of `/master`

Comments: