noamaanMulla-03/transformers-issues huggingface dataset:
Deepspeed OOM When training 7B model on V100 16GB (2)
Author: shahules786
Pull request: no

### System Info

 - Python version 3.8
- transformers - installed from source latest

**Describe the bug**
OOM When training 7B model on V100 16GB (2) with zero stage 2 and CPU offloading even when memory estimation showed far less per GPU mem requirement. 
```
-- memory estimation--
DEVICES ['Tesla V100-PCIE-16GB', 'Tesla V100-PCIE-16GB']
-------------ZERO 2------------
Estimated memory needed for params, optim states and gradients for a:
HW: Setup with 1 node, 2 GPUs per node.
SW: Model with 6650M total params.
  per CPU  |  per GPU |   Options
  148.66GB |  12.39GB | offload_optimizer=cpu
   74.33GB |  74.33GB | offload_optimizer=none
```

**Screenshots**
nvidia-smi during run
<img width="648" alt="Screenshot 2023-06-19 at 5 30 41 PM" src="https://github.com/microsoft/DeepSpeed/assets/25312635/47a512c6-b509-49c2-b111-8f7e9dac8532">

RAM usage
<img width="723" alt="Screenshot 2023-06-19 at 5 55 08 PM" src="https://github.com/microsoft/DeepSpeed/assets/25312635/e7e97e07-7203-42a7-8b89-468c2de35546">
Can see free RAM available.

**System info (please complete the following information):**
 - OS: CentOS Linux
 - GPU count and types : V100 16B X 2 single node



### Who can help?

@pacman100 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Steps to reproduce the behavior:
My code is [here](https://github.com/explodinggradients/Funtuner/blob/main/funtuner/trainer.py)
Run deepspeed funtuner/trainer.py

export PYTHONPATH="${PYTHONPATH}:/your-path/Funtuner"
please change the log_dir to your folder [here](https://github.com/explodinggradients/Funtuner/blob/c4e66209d5ee276a7eb8caf582435f1eaafbf18f/funtuner/config/config.yaml#L4) also you might want to set log_wandb=False
`dev-train` branch
 

### Expected behavior

Run w/o OOM error.


Comments:
Did you solved?
fix set_transform link docs
Author: diegulio
Pull request: yes

# What does this PR do?

This PR simply fix a link to set_transform documentation. 

<!-- Remove if not applicable -->

Fixes # (issue)

Broken link to set_transform docs

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu  @MKhalusova 






Comments:
Simplifying getting the number of embedding tokens
Author: AnastasiyaKukharska
Pull request: yes

# What does this PR do?

Here I simplify getting the number of embedding tokens according to discussions on PR [26024](https://github.com/huggingface/transformers/pull/26024)

# Before submitting

- [ ]  This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x]  Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ]  Did you make sure to update the documentation with your changes? Here are the [documentation guidelines (https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ]  Did you write any new necessary tests?



Comments:
Hello, this is a duplicate of #26333 and will thus not be merged
[WIP] Make logic of adding the number of embedded tokens simpler
Author: AnastasiyaKukharska
Pull request: yes

# What does this PR do?

Here I simplify getting the number of embedding tokens according to discussions on PR [26024](https://github.com/huggingface/transformers/pull/26024)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
Fix Fuyu image scaling bug
Author: pcuenca
Pull request: yes

The coordinates are reversed, see https://github.com/huggingface/transformers/blob/574a5384557b1aaf98ddb13ea9eb0a0ee8ff2cb2/src/transformers/image_transforms.py#L328

It could produce negative padding and hence inference errors for certain image sizes, as [identified in the Fuyu demo](https://huggingface.co/spaces/adept/fuyu-8b-demo/discussions/1)

Happy to write a small test in the morning.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@molbap @ArthurZucker @LysandreJik 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

~~Thanks for the quick review! Feel free to merge it when you see fit, as I can't do it :)~~ Edit: saw that you requested a review from Pablo 👍 

He might have missed it let's merge this 😉
Updating `build-docker-images.yml` to build AMDGPU containers
Author: mfuntowicz
Pull request: yes

This PR adds the following:
- [x] Job to build AMD specific docker images for the CI
- [x] Update the GA docker dependencies to their latest version(s)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26262). All of your documentation changes will be reflected on that endpoint.

I think this could be closed as the changes is already on `main`? cc @mfuntowicz 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
the NoRepeatNGramLogitsProcessor is slow
Author: nevakrien
Pull request: no

### Feature request

its relatively easy to just cash the values ahead of time and then use those, rn its build in a way that requires looking through the whole text every single time. u could have simply kept the dict and added 1 value to it.

### Motivation

I am trying to make generation faster

### Your contribution

I could probably help with the pull request if you guys r interested 

Comments:
cc @gante 

Hey @nevakrien 👋 

Our main focus is the breadth and availability of techniques, which means performance sometimes gets relegated to a secondary position. Your contribution (with a few benchmarks before/after the changes) would definitely be appreciated 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
A potential bug in convert_llama_weights_to_hf.py
Author: lumosity4tpj
Pull request: no

### System Info

I did not run the hf code, just found a potential bug while referring to the code

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I found a potential bug in [convert_llama_weights_to_hf_L114](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py#L114), the cause of the error is that `num_key_value_heads` is used as` num_key_value_groups`, the reason for the success of the` llama2 `conversion weights, Because `num_key_value_heads=num_key_value_groups=8`,

A simple example would be: `num_key_value_heads=32, num_heads=64, num_shards=8`, then n_heads_per_shard = n_heads // num_shards=8 `, num_local_key_value_heads = n_heads_per_shard // num_key_value_heads = 0 `

`num_key_value_heads` should, as I understand it, refer to the total number of` kv_heads`, See [this](https://github.com/facebookresearch/llama/blob/main/llama/model.py#L186),

### Expected behavior

just correct

Comments:
Hi @lumosity4tpj 
Thanks a lot for your message. Your explanation seems to makes sense, note we used this script internally to convert Llama weights into HF format and it worked, we probably did not supported all possible edge cases but the conversion script would work for the models that are currently on the Hub. Maybe @ArthurZucker has more context for this specific issue

Feel free to open a PR with a fix if you want (mathematically yes, we would have a fail) but no checkpoints were released with `num_key_value_heads != 8`. This is also related to performance optimisation, as you would have more copies to make locally if `num_key_value_heads != num_key_value_groups`.

> just correct

it's already correct with released checkpoints

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix benchmark tests on RoCm
Author: fxmarty
Pull request: yes

Some benchmark tests (not sure if they are commonly run) do not pass on RoCm systems due to py3nvml being nvidia-specific. This PR fixes the issue using `rocm-smi` instead, I couldn't find a binding to an equivalent of NVIDIA Management Library for AMD.

Those benchmark tests may be remove in favor of the optimum-benchmark built by @IlyasMoutawwakil though, I guess

We may need a `torch-dev-rocm` to avoid installing pynvml on RoCm systems.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_25990). All of your documentation changes will be reflected on that endpoint.

@fxmarty What about using the rocm wrapper? Might be something cleaner moving forward:

https://github.com/RadeonOpenCompute/rocm_smi_lib/blob/master/python_smi_tools/rocm_smi.py

@mfuntowicz There is also https://github.com/RadeonOpenCompute/pyrsmi, unfortunately it appears that this package does not have a pypi release on pypi index yet.

Resolving dependencies could be done by editing transformers `setup.cfg` to add the index `https://test.pypi.org/simple`, but it seems quite risky given that many packages may push to pypi test as well. 

I am not sure the source you pointed to would help either, it is not exposed through a python package. So I guess we would need to modify our path (ugly) or run subprocess anyway.

For completeness, fixes:

```
[gw9] [  2%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_encoder_decoder_with_configs
[gw9] [  4%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_no_configs 
tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain
[gw9] [  4%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_no_configs_only_pretrain 
tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_no_model_no_architectures 
[gw9] [  6%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_torchscript 
[gw9] [  7%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_inference_with_configs 
[gw9] [  7%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_save_csv_files 
[gw9] [  9%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_trace_memory 
tests/benchmark/test_benchmark.py::BenchmarkTest::test_train_encoder_decoder_with_configs 
[gw9] [ 11%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_train_no_configs 
[gw9] [ 13%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_train_no_configs_fp16 
[gw9] [ 14%] FAILED tests/benchmark/test_benchmark.py::BenchmarkTest::test_train_with_configs 
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
DetrImageProcessor.normalize_annotation() got an unexpected keyword argument 'input_data_format'
Author: r-remus
Pull request: no

### System Info
- `transformers` version: 4.32.0
- Platform: Linux-5.15.0-79-generic-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>



### Who can help?
@amyeroberts



### Information

- [ ] The official example scripts
- [X] My own modified scripts



### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)



### Reproduction

`DetrImageProcessor.normalize_annotation()` doesn't take the new `input_data_format` which is passed to it.

Code to reproduce the issue:
```python
from transformers import DetrImageProcessor
image_processor = DetrImageProcessor()
_ = image_processor(images=[img], annotations=[{'image_id': 0, 'annotations': []}], return_tensors="pt")
```


### Expected behavior

It should be possible to pass `input_data_format` to `DetrImageProcessor.normalize_annotation()`.

Comments:
Are you using main? This is a duplicate of #25666 and should have been fixed by #25643

@r-remus The fix has now been included as part of a [patch release](https://github.com/huggingface/transformers/releases/tag/v4.32.1) and can be directly installed from pypi with `pip install transformers`

Sorry for getting back to you so late.
@ArthurZucker: No, I wasn't using main but release v4.32.0.
@amyeroberts: Thanks, that's great! Works for me now (using v4.33.2 now, tho).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
The impact of a quantization config on .num_parameters
Author: BramVanroy
Pull request: no

### System Info

- `transformers` version: 4.32.1
- Platform: Linux-5.14.0-284.25.1.el9_2.x86_64-x86_64-with-glibc2.34
- Python version: 3.11.4
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.3
- Accelerate version: 0.22.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)


### Who can help?

@SunMarc and @younesbelkada

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I am trying to understand the impact of a quantization config on the number of parameters better but as discussed internally I am not sure whether what I am experiencing is a bug in `num_parameters` or the expected behavior when using quantization. 

As far as my understanding of quantization goes, it usually implies **changing the precision** of parameters but not specifically reducing the number of parameters. However, when I run the following code, I can see that the number of parameters is cut in half (~6B compared to 13B).


```python
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype="bfloat16",
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-13b-chat-hf", quantization_config=bnb_config, device_map="auto")
model.num_parameters()
# 6671979520
```

### Expected behavior

Is this behavior expected? If so, what is the reason/theory behind it? If not, can the `num_parameters` implementation be fixed?

Comments:
I tried with `llama-7b`...
1. w/o 4 bit quantization: (16 bit)
trainable params: 33,554,432 || all params: 6,771,970,048 || trainable%: 0.49548996469513035
2. w/
trainable params: 33,554,432 || all params: 3,533,967,360 || trainable%: 0.9494833591219133

I wonder why changing the precision leads to the decrease of the parameters...

Is this behavior expected?

Hi @BramVanroy 
This is indeed a bug and needs fixing! Please see #26132 that explains the fix

Cool, thanks! @younesbelkada 

I think this behavior might be due to the implementation of Linear4bit: https://github.com/TimDettmers/bitsandbytes/blob/main/bitsandbytes/nn/modules.py

`requires_grad=False` is always set for weights
Fix no_grad when using AdaFactor out of transformers.
Author: richarddwang
Pull request: yes

When using `transformers.optimization.AdaFactor` with my own code not using `Trainer`, autograd will say there is no grad to backprop. 

I believe it might work well with `Trainer`, but this simple and small fix makes it more useful.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Segmentation Fault when trying to load HuggingFace Model
Author: y12uc231
Pull request: no

### System Info

```
Python 3.9.12 (main, Apr  5 2022, 01:53:17) 
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import AutoModel
>>> model = AutoModel.from_pretrained("gpt2")
zsh: segmentation fault  python
```

I am using transformers version = '4.33.1'


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In the python terminal. 
```
Python 3.9.12 (main, Apr  5 2022, 01:53:17) 
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import AutoModel
>>> model = AutoModel.from_pretrained("gpt2")
zsh: segmentation fault  python
```

### Expected behavior

The model should load instead of throwing a segmentation fault. 

Comments:
Hey! I cant reproduce this, you should try to re-install python this does not seem to have anything to do with transformers

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix logit-to-multi-hot conversion in example
Author: ranchlai
Pull request: yes

# What does this PR do?

This PR fixes #26830 . The bug might lead to difference in measuring multi-label acc (Slightly higher precision and lower recall). 

@younesbelkada Would you please review. 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26936). All of your documentation changes will be reflected on that endpoint.

@ArthurZucker I've updated the comment for educational purpose.