noamaanMulla-03/transformers-issues huggingface dataset:
🚨🚨🚨 Replace DataLoader logic for Accelerate in Trainer, remove unneeded tests 🚨🚨🚨
Author: muellerzr
Pull request: yes
Labels: name: External

name: Tests

name: Distributed Training / Models

name: trainer

# What does this PR do?

This PR:
- Guts the internals for the `DataLoader` in all basic distributed fashions (replacing `pl.Loader` for TPU coming in a follow-up PR) to replace it with `accelerator.prepare`
- Removes **two** tests that were deemed unnecessary
  - Test 1 removed: `tests/trainer/test_trainer.py::TrainerIntegrationTest::test_sampler_seed`, deemed to no longer be necessary to reset the seed, as Accelerate's dataloader setup doesn't need any extra help when iterating/loading back in the seed, regardless of the torch version
  - Test 2 removed: `tests/trainer/test_trainer.py::TrainerIntegrationTest::test_training_finite_iterable_dataset`, as with Accelerate's new sampler for `IterableDataset` we'll actually catch if it's `None` and raise an error, a new test will be made + clear error message on the `Accelerate` side, with a test added to `Trainer` afterwards. 
- Modifies two tests to use the proper attribute: Accelerator's `DataLoaders` all have `total_batch_size` rather than `batch_size`
  - `test_train_and_eval_dataloaders` and `test_data_is_not_parallelized_when_model_is_parallel`


Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger @pacman100 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Great PR, currently this is breaking my custom collate_fn in the dataloader, still trying to understand why that is. First assumption might be due to multiprocessing?

@franz101 please open an issue with a reproducer of what you are trying to do so we can help :)
NotImplementedError: offload_to_cpu=True and NO_SHARD is not supported yet
Author: linkailuo1986
Pull request: no
Labels: name: solved

### System Info

I was using fsdp with settings "full_shard auto_wrap" on a A100 GPU. The training went well but was interupted when saving the checkpoints. The error stated `NotImplementedError: offload_to_cpu=True and NO_SHARD is not supported yet`. I understand that I am using a single GPU so fsdp defaluts to NO_SHAPR. However, I dont understand why offload_to_cpu was set to True. Or anywhere I can reset it to false?

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

following https://github.com/lm-sys/FastChat to fine-tune an LLM

### Expected behavior

the error as stated.

Comments:
Can you please give us a code reproducer of the issue? cc @pacman100 

> Can you please give us a code reproducer of the issue? cc @pacman100

Thanks. Sure, here is the code:

`torchrun train.py \
    --model_name_or_path openlm-research/open_llama_3b  \
    --data_path /path/to/data \
    --bf16 True \
    --output_dir /path/to/output \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 4 \
    --evaluation_strategy "no" \
    --eval_steps 1500 \
    --save_strategy "steps" \
    --save_steps 2 \
    --save_total_limit 3 \
    --learning_rate 2e-5 \
    --weight_decay 0. \
    --warmup_ratio 0.04 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --fsdp "shard_grad_op auto_wrap" \
    --fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
    --tf32 True \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True
`

`train.py` can be found from [here](https://github.com/lm-sys/FastChat/blob/main/fastchat/train/train.py)
You can use dummy data from [here](https://github.com/lm-sys/FastChat/blob/main/data/dummy_conversation.json)

Hello, thank you @linkailuo1986, the above PR should fix it. But it doesn't make sense to use FSDP on a single GPU

Thanks @pacman100 for the quick fix. I used FSDP because it seemd to reduce VRAM for a larger batch size, which otherwise got an OOM error without using it.
To work out tokenization_utils_base.py:731 list to tensor so slow
Author: askxiaozhang
Pull request: yes

tokenization_utils_base.py:731: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.asarray() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:230.)
Solved this problem to accelerate  list to tensor.

Comments:
@askxiaozhang Thanks for opening this PR and contributing to improving the transformers library! There is another PR opened #24772 which addresses this, and so this PR will not be merged in.
[WIP] Add LaVIN
Author: shauray8
Pull request: yes

# What does this PR do?
Adds the LaVIN model from - https://arxiv.org/pdf/2305.15023.pdf <br>
Model description - LaVIN is a vision-language instructed model that is affordable to train (it was trained in a few hours on 8 A100 GPUs) with good performance on ScienceQA.

Fixes issue #23846

## Who can review?

Models:
@amyeroberts @ArthurZucker 

** Draft ** (Maintainers and reviewers can go through the PR as and when needed, I will ping the reviewers once the PR is ready. Guidance/Questions/Concerns related to the PR are always welcome.

Comments:
Hi @shauray8, thanks for opening this PR! 

The easiest and recommended way to make a model available in `transformers` is to add the modeling code directly on the hub: https://huggingface.co/docs/transformers/custom_models

This means, once working, the model can be found and used immediately without having to go through the PR process. We find this is a lot quicker as the bar for adding code into the library is high due to the maintenance cost of every new model, and so reviews take quite a while.

Hi @amyeroberts, That makes sense, I have not seen a lot of people use this particular model. I'll make all the necessary changes and add it to the hub. But if there's anything I can help with to improve HuggingFace I'm more than happy to do it.
Add support for Llama-2-70b-chat-hf in transformers
Author: Daryl149
Pull request: no
Labels: name: New model

### Model description

Not sure if it is a bug, or that it is intentionally not supported yet. In either case: there have been 0 confirmations of people being able to successfully run the official **Llama-2-70b-chat-hf** model in transformers.

### Open source status

- [ ] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Official model weights:
https://huggingface.co/meta-llama/Llama-2-70b-chat-hf
Related open bug:
https://github.com/facebookresearch/llama/issues/423

Comments:
Turns out the model is supported, but has a bug in the `config.json`, see https://github.com/facebookresearch/llama/issues/423
Also, I am adding a version that works without any manual changes here: https://huggingface.co/daryl149/llama-2-70b-chat-hf
T5 Tokenizer Adds Space after Each Added (Extra) Token
Author: tshu-w
Pull request: no

### System Info

- `transformers` version: 4.30.2
- Platform: Linux-5.4.0-146-generic-x86_64-with-glibc2.35
- Python version: 3.11.3
- Huggingface_hub version: 0.15.1
- Safetensors version: 0.3.1
- PyTorch version (GPU?): 2.0.1+cu117 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: (NA)
- Using distributed or parallel set-up in script?: (NA)


### Who can help?

@Arthu

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```ipython
In [1]: from transformers import AutoTokenizer

In [2]: tokenizer = AutoTokenizer.from_pretrained("./models/t5-base/")

In [3]: tokenizer.add_tokens(["asdfg"], special_tokens=False)
Out[3]: 1

In [4]: tokenizer.tokenize("asdfgwordtimeasdfgtime")
Out[4]: ['asdfg', '▁word', 'time', 'asdfg', '▁time']
```

### Expected behavior

tokenizer return `['asdfg', 'word', 'time', 'asdfg', 'time']`

Comments:
I think a fix is in

https://github.com/huggingface/transformers/pull/24622



FYI: that PR is not merged yet into `main` branch

Let's wait until we merge to close! 

@ArthurZucker  Hi, this issue still exists after updating transformers to the latest 4.31.0 with #24622

Hey! 
It is adressed for slow tokenizer, which are part of transformers! Fast tokenizers will need to wait a bit. It is also linked to the conversion script and meta space that need to be used similar to Llama
```python 
In [6]: tokenizer = AutoTokenizer.from_pretrained("t5-base", legacy = False, use_fast = False)
/fsx/arthur/miniconda3/envs/py10/lib/python3.10/site-packages/transformers/models/t5/tokenization_t5.py:199: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.
- Be aware that you SHOULD NOT rely on t5-base automatically truncating your input to 512 when padding/encoding.
- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.
- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.
  warnings.warn(

In [7]: tokenizer.add_tokens(["asdfg"], special_tokens=False)
Out[7]: 1

In [8]: tokenizer.tokenize("asdfgwordtimeasdfgtime")
Out[8]: ['asdfg', 'word', 'time', 'asdfg', 'time']
```
the key is that you need to set `legacy=False` and `use_fast = False` because fast tokenizer is not fixed yet 😉
BatchSampler
Author: Neptune-Trojans
Pull request: no

### Feature request

I am trying to use Transformers Trainer and I want to generate batches and not single items. 


### Motivation

I need that in order to generate batches of different sizes without doing padding.
Is that possible to do with the implemented trainer ?

### Your contribution

..

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions as we keep issues for bugs and feature requests only.
Update processing_vision_text_dual_encoder.py
Author: premsa
Pull request: yes



# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes: small typo: kwrags -> kwargs


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).



## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24950). All of your documentation changes will be reflected on that endpoint.
Deprecate unused OpenLlama architecture
Author: tomaarsen
Pull request: yes

Hello!

# What does this PR do?
1. Deprecate `OpenLlama` following #24787.
2. Add a disclaimer pointing users to `LLaMA` for the [OpenLLaMA models](https://huggingface.co/models?search=openllama).
3. Resolve a typo in a warning in `check_repo.py`
4. Read modeling files with `encoding="utf8"` in `check_config_attributes_being_used`.

If preferred, I can revert 3 and 4 and cherry-pick them into a separate PR.
Follow-up of #24913. This is a considerably better solution - I feel a bit embarrassed I even considered the other one.

## Details
I've followed the steps that @sgugger seems to have taken from #24787 to move OpenLlama into deprecation. This involves moving the main code, adapting the `__init__` files, removing the tests, and updating the documentation with a disclaimer.

Feel free to let me know if you'd rather keep the model non-deprecated for the time being, and then I'll revert to only the addition of the disclaimer.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Discussed in #24913.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 

- Tom Aarsen


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> FAILED tests/test_modeling_utils.py::ModelPushToHubTester::test_push_to_hub - huggingface_hub.utils._errors.HfHubHTTPError: 500 Server Error: Internal Server Error for url: https://hub-ci.huggingface.co/api/repos/create

Test failure seems unrelated, I can push an empty commit or let you rerun them.

> Re-launched the tests to make sure it's just a fluke. Pinging the original author @s-JoL for information and to see if there is any plan on open-sourcing another checkpoint for this architecture?

Due to various reasons, the previous open-source project has been shut down. Considering that there are more and more open-source models available now (including commercially available ones like Llama2), I believe that open-sourcing another similar model won't add much value to the community. Therefore, I think it's reasonable to mark this model as deprecated. However, I understand that some users are training with the Llama model that includes XFormers. Is it possible to add an optional XFormers acceleration in the Llama model to facilitate these users?

We can see how to add an optional XFormers acceleration in Llama in other PRs, yes.
Incorrect typing of `fsdp`, `fsdp_config`, and `sharded_ddp` in `TrainingArguments`
Author: O-T-O-Z
Pull request: no

### System Info

- `transformers` version: 4.29.2
- Platform: macOS-14.0-arm64-arm-64bit
- Python version: 3.10.6
- Huggingface_hub version: 0.15.1
- Safetensors version: not installed
- PyTorch version (GPU?): 1.13.0.dev20220902 (False)
- Tensorflow version (GPU?): 2.9.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

When initializing a `pydantic.BaseModel` as follows:
```python
from pydantic import BaseModel
from transformers.training_args import TrainingArguments

class MyTrainingArguments(TrainingArguments):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        self.my_arg = "my_arg"


class MyModel(BaseModel):
    training_args: MyTrainingArguments


model = MyModel(training_args=MyTrainingArguments(output_dir=""))
```
The following `ValidationErrors` occur:
```shell
ValidationError: 4 validation errors for MyModel
training_args -> debug
  str type expected (type=type_error.str)
training_args -> sharded_ddp
  str type expected (type=type_error.str)
training_args -> fsdp
  str type expected (type=type_error.str)
training_args -> fsdp_config
  str type expected (type=type_error.str)
```

Since `debug` has been fixed in #24033, my main concern are the others.
After investigation, I discovered that the `__post_init__()`-method changes these parameters from their default `str` values to for example `dict`, `bool`, or `List`.  This becomes a problem for Pydantic (and other type-checkers) since the validation will be incorrect, while the docstring of `TrainingArguments` describes the following for these parameters:
```python
"""
sharded_ddp (`bool`, `str` or list of [`~trainer_utils.ShardedDDPOption`], *optional*, defaults to `False`)
fsdp (`bool`, `str` or list of [`~trainer_utils.FSDPOption`], *optional*, defaults to `False`)
fsdp_config (`str` or `dict`, *optional*)
"""
```

### Expected behavior

I would like to resolve these issues by providing the correct typehinting. This could look as follows:
```python
sharded_ddp: Union[Optional[str], bool, List[ShardedDDPOption]]
fsdp: Union[Optional[str], bool, List[FSDPOption]]
fsdp_config: Union[Optional[str], Dict]
```
I checked this configuration and it resolves the issue.

Comments:
We do not support any automatic type checker and our type annotations are only here to give documentation. We welcome PRs to make them more exact as long as it's not at the cost of code readability but the bottomline is that you shouldn't use a type-checker with hard errors on Transformers.

@sgugger So should I create a PR for this?
[DOCS] Example for `LogitsProcessor` class
Author: shauray8
Pull request: yes

# What does this PR do?
Added some doc string to `RepetitionPenaltyLogitsProcessor` with some examples as well.
@gante let me know if there's anything else I should add or remove from the docs.

Fixes # (issue)
#24783

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

Comments:
You'll probably need to run `make fixup` before you next commit :)

Previously I was having some problems with `make fixup` but now it's done and reformatted, I really like how the Makefile is structured. 

@sgugger note: the weird part of the diff seems innocuous 🤔 

_The documentation is not available anymore as the PR was closed or merged._

@sgugger I have addressed all the suggested changes. 

Thanks!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24848). All of your documentation changes will be reflected on that endpoint.
fix type annotations for arguments in training_args
Author: shauray8
Pull request: yes

# What does this PR do?

Fixes #24538 which is basically fixing type annotations for `fsdp`, `fsdp_config`, and `sharded_ddp` in training_args.py

## Who can review?
maybe @sgugger 

Comments:
Fixing CI errors! 

_The documentation is not available anymore as the PR was closed or merged._

@sgugger I don't particularly understand why this error occurs in examples_flax - ``` argparse.ArgumentError: argument --sharded_ddp: invalid typing.Union[str, bool, typing.List[transformers.trainer_utils.ShardedDDPOption], NoneType] value: '' ```

@sgugger `bool` breaks `--sharded_ddp`, I think we can still maintain Boolean arguments with string itself and 
https://github.com/huggingface/transformers/blob/20d6b84613984f2497587a62774704882ccbeee6/src/transformers/hf_argparser.py#L168-L173
with this `--sharded_ddp` and `--fsdp` defaults to string 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_24550). All of your documentation changes will be reflected on that endpoint.
About finetuning Whisper by multi-GPUs
Author: LYPinASR
Pull request: no

### Feature request

I want to finetune Whisper by 4 GPUs, what should I do?

### Motivation

finetuning Whisper by multi-GPUs

### Your contribution

finetuning Whisper by multi-GPUs

Comments:
Hi @LYPinASR, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.