noamaanMulla-03/transformers-issues huggingface dataset:
Update checks for torch.distributed.tensor to require torch >= 2.5
Author: loadams
Pull request: yes

# What does this PR do?

Supplements #34800, the checks for torch.distributed.tensor require torch >= 2.5 as the comment states, this adds the check for it and updates the check.

Fixes #34795

## Who can review?

@ArthurZucker @muellerzr

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34816). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for this! I tried these fixes but I ran into this error:
```
ImportError: cannot import name 'Replicate' from 'torch.distributed.tensor' (/usr/local/lib/python3.10/dist-packages/torch/distributed/tensor/__init__.py)
```
when using torch 2.5.0, which was issued from [this line](https://github.com/huggingface/transformers/blob/e1f6ff7d9183e45246d08e07703465a9210191b0/src/transformers/pytorch_utils.py#L44). I was able to make it work with torch 2.5.1.

@pglorio imports work fine for me with `torch==2.5.0`

> Thanks, let's rather use `is_torch_greater_or_equal("2.5")` from
> 
> https://github.com/huggingface/transformers/blob/6aba68c0b9508b8c8fce025bd5492151258cd57a/src/transformers/utils/import_utils.py#L932-L938
> 
> And good to go!

Thanks @ArthurZucker - I've made that change but left the new function I added for checking torch>= 2.5, though it probably makes sense to refactor to not use those functions in the future.

The tests look to be unrelated failures though?



> LGTM ! I reran the tests. These are unrelated to the PR

Thanks @SunMarc 

THanks, merging!
Remove quantization related config from dequantized model
Author: konradkalita
Pull request: yes

# What does this PR do?

Fixes #34847


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

 @SunMarc


Comments:
What's going on with T5 x torch.compile ?
Author: kanpuriyanawab
Pull request: no
Labels: name: WIP

name: bug

name: Compilation

### System Info

Hi Team,
First of all huge thanks for all the great work you are doing.

Recently, I was benchmarking inference for T5 model on ‪AWS EC2 ( G6E machine with L40 GPU) for batch sizes of 1, 2, 4.

I have heard tons about torch. compile and wanted to try it out and see if it reduces the inference time. Surprisingly, it did the other way around. On average, I saw an increase of ~1 sec in inference time for a sample size of 50 with a length of each sample ranging from [2200, 3000] characters, with an average of around 2550 chars.

I had a chat with a friend about this who told me that T5 is not a very suitable architecture for compilation yet and there are lots of graphbreaks. With his advice, I decided to open an issue here. 

From my experience, T5 is still a very good model and I would want to see it work seamlessly with torch compile. If chance comes, I am ready to put my own time into this and contribute to the cause. Let me know what you think.

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

 ‪AWS EC2 ( G6E machine with L40 GPU) for batch sizes of 1, 2, and 4.

### Expected behavior

The inference time should reduce post compilation.

Comments:
Thanks for the issue and feature request @shivance! 

cc @ArthurZucker regarding supporting torch.compile for T5.

Hey! T5 does not support the new "cache_positions" so `generation` will probably be slow as it has to deal with `dynamic shapes`. 

> T5 is not a very suitable architecture for compilation 

I completely disagree with that! 😉 we just did not have time to ship compile for this model. Tho #32617 should give you alead and also #31166

T5 and BART are planned to be compile-compatible in the next batch of models, as it is an encoder-decoder model. I will work on it next month if there's no PR by that time 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

The T5 model is compile compatible now, so closing as resolved 

@shivance did you by any chance benchmark the new T5 code? Any perf improvements?
Urgent: 429 Errors and Potential Rate Limiting on Model Access
Author: dnuanghsh
Pull request: no
Labels: name: bug

### System Info

Hi team,

From yesterday, we have been seeing 429 errors when trying to access different models.


Here's an example of the error we're seeing:

```
File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 1024, in raise_for_status  
    raise HTTPError(http_error_msg, response=self)  
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for URL: https://huggingface.co/meta-llama/Meta-Llama-3-8B/resolve/main/config.json  
```

We suspect we are being rate-limited. We can share the IPs with you.

Could we look into resolving this issue? 

Thank you so much!

cc @amyeroberts @philschmid  @sgugger 


### Who can help?

@amyeroberts @philschmid @sgugger 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

We ran into the issues in our internal runs. 

### Expected behavior

Run without hitting the 429 errors.

Comments:
Hi @dnuanghsh, can you tell us how many requests were being made simultaneously?

Hi @Rocketknight1, there was a sudden surge in the number of requests due to an integration test (reverted) on 11/19. Our normal workload never encountered this issue. Is it possible for us to follow a similar pattern as mentioned in https://github.com/huggingface/transformers/issues/21129 to resolve the issue? Thank you so much!

Update: It looks like the rate limiting has been lifted.

Could you please share more information on the pattern of rate limiting? What typically causes rate limiting? Additionally, do you have any best practices for working with huggingface to avoid encountering this issue in the future? Thank you!

@Rocketknight1 Do you have any suggestion to avoid this issue in the future? Like any business account to relieve this restrict? Thanks

Hi @dnuanghsh glad to hear it's working for you now. Can you please email api-enterprise@huggingface.co with your IP address so that we can investigate why this happened?

@xyg-coder Feel free to email us at api-enterprise@huggingface.co as well, as well as any information about when you received the rate limiting error message/what you were doing.

@meganriley We have emailed you. Thank you!

Handling via email, closing issue
DETR preprocess with a wrong BBox format
Author: junxnone
Pull request: no
Labels: name: Vision

### System Info

main branch

### Who can help?

@amyeroberts @qubvel 

GroundingDINO have the same issue @EduardoPach 


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

- 1 **The original DETR bbox format**:
  - Input BBox Format: [`(x, y, x, y)` => `(cx, cy, w, h)` ](https://github.com/facebookresearch/detr/blob/29901c51d7fe8712168b8d0d64351170bc0f83e0/datasets/transforms.py#L254-L256))
  - Calculate BBox Loss with `(cx, cy, w, h)` 
  - Output Convert [to `(x, y, x, y)`](https://github.com/facebookresearch/detr/blob/29901c51d7fe8712168b8d0d64351170bc0f83e0/models/detr.py#L277-L278)
- 2 **transformers DETR Model** 
  - Input BBox as **COCO Format** `(x, y, w, h)`, and convert to `(x, y, x, y)` [here](https://github.com/huggingface/transformers/blob/6a912ff2c5b3eadb9a0583d77083aae27d35d28d/src/transformers/models/detr/image_processing_detr.py#L351-L353)
  - **Then** calculate the bbox loss with **Format Convert** [center_to_corner](https://github.com/huggingface/transformers/blob/6a912ff2c5b3eadb9a0583d77083aae27d35d28d/src/transformers/loss/loss_for_object_detection.py#L177-L179), **but the target bbox is still (x, y, x, y).** 
- 3 The **GroundingDINO** model have the same **input BBox Format** issues
  - And the **Open-GroundingDINO** also convert [(x,y,x,y) to (cx,cy,w,h)](https://github.com/longzw1997/Open-GroundingDino/blob/924bb6c4b93cae2dae582e1afaeccd408c72a31d/datasets/transforms.py#L264)
  - But I am not sure why the **original GroundingDINO** model output the [(xywh) format](https://github.com/IDEA-Research/GroundingDINO/blob/856dde20aee659246248e20734ef9ba5214f5e44/demo/inference_on_a_image.py#L31)
- 4 not sure if this is the root cause of #28865

### Expected behavior

The model expected the bbox format for loss calculate is `(cx,cy,w,h)`

Comments:
Hi @junxnone! Thanks for opening the issue! Bbox conversion is indeed the trickiest part of our pipeline. You can find some detailed explanations in this issue:

 - https://github.com/huggingface/transformers/issues/32835

It’s definitely an area that could be improved!

Oops, Found the [xyxy2cxcy format convert](https://github.com/huggingface/transformers/blob/d9e6f307e71b5108a7882ec00ffcc0d0eb316cb7/src/transformers/models/detr/image_processing_detr.py#L227-L238), it is same as the original DETR, do convert in the Normalize function.

Thanks.
VitMaePreTrain Example norm_pix_loss arg set default to False
Author: mikecummings34
Pull request: yes

# What does this PR do?
Changes will set the default value of ```norm_pix_loss``` to ```False``` in ```run_mae.py``` and  explicitly set ```--norm_pix_loss``` to ```False``` in the example script.  This is a workaround for issue #29416 until the real problem can be solved in reference to the VitMae whitepaper suggesting to normalize pixel values. However, it seems that there is a deeper issue causing pixel outputs to be abnormal.




Comments:
cc @nielsrogge
Skip eetq test if it attempts to import shard_checkpoint
Author: MekkCyber
Pull request: yes

# What does this PR do?
PR inspired from peft by @BenjaminBossan https://github.com/huggingface/peft/pull/2226. 

EETQ attempts to import the shard_checkpoint function from the transformers library, but this function has been removed in the latest version. As a result, using EETQ currently causes an import error, leading to all tests failing. This fix ensures that EETQ tests are skipped if an import error occurs.

The issue is reported to EETQ: NetEase-FuXi/EETQ#34.

## Who can review  ?
@SunMarc 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34868). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I have already started a PR: #34854. Sorry for the missing communication.

No problem will close this one then
Unable to import SamModel
Author: leemorton
Pull request: no
Labels: name: bug

### System Info

transfomers=4.46.3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi, I am unable to import SamModel, SamProcessor works fine but as soon as i add SamModel i get the following error...

RuntimeError: Failed to import transformers.models.sam.modeling_sam because of the following error (look up to see its traceback):
Failed to import transformers.generation.configuration_utils because of the following error (look up to see its traceback):
cannot import name 'Replicate' from 'torch.distributed.tensor' (/home/jupyter-lmorton/.local/lib/python3.10/site-packages/torch/distributed/tensor/__init__.py)

```
! pip3 install git+https://github.com/facebookresearch/segment-anything.git

! pip3 install opencv-python pycocotools matplotlib onnxruntime onnx boto3 torch torchvision datasets lxml patchify monai transformers
```

```python
from segment_anything import SamAutomaticMaskGenerator, SamPredictor, sam_model_registry
from transformers import SamModel, SamProcessor
```

Could there be something I am doing wrong?

I am following this example:
https://github.com/bnsreenu/python_for_microscopists/blob/master/331_fine_tune_SAM_mito.ipynb

Thanks

### Expected behavior

import works

Comments:
Hi @leemorton! thanks for reporting it! You can either update torch to 2.5 (or downgrade to 2.3) or install the previous version of transformers. The fix is on the way 🤗 

> Hi @leemorton! thanks for reporting it! You can either update torch to 2.5 (or downgrade to 2.3) or install the previous version of transformers. The fix is on the way 🤗

Thanks @qubvel , it gave a different error relating to transformers.models.sam.processing_sam trying to import InterpolationMode in torchvision.transforms after trying these versions:

torch==2.5.0 torchvision==0.2.0
torch==2.3 torchvision==0.18
torch==2.2 torchvision==0.17

Also gave it a go with:
transformers==4.44.0
transformers==4.43.0
transformers==4.42.0

![image](https://github.com/user-attachments/assets/b741f02c-63a5-4a66-81ee-ac689b236d08)


Hmm.. here is my setup that works fine

```
- `transformers` version: 4.47.0.dev0
- PyTorch version (GPU?): 2.5.0+cu118 (True)
- torchvision version: 0.20.0+cu118
- Pillow version: 10.4.0
```

Please make sure the correct version of `torchvision` installed (0.2.0 seems too old)

Thanks I restarted my kernel and 2.5 + 0.20 + 4.47 worked :)

Should be fixed with 
 - https://github.com/huggingface/transformers/pull/34816
smol improvements to support more flexible usage
Author: andimarafioti
Pull request: yes
Labels: name: Vision

name: Processing

# What does this PR do?

This PR improves a few smol issues we had with Idefics 3:

1) We couldn't use the model with images larger than 5*364. This was the default max_image_size. The method where this was computed took a parameter as input, but it was never used. It would also raise an error if we wanted to resize to a larger size. I changed this for a default value of 4k resolution, as this is already considerably larger than what we trained on, ie, anything larger is pretty outrageous.
2) We couldn't train with datasets that contained grayscale images since the input_data_format wasn't properly parsed. I fixed this by switching around the processing order. Now, if the images are grayscale, I add a channel to the end or start of the images. Then, the input_data_format can be correctly inferred if it is none. 
3) Finally, when converting to pil_image, we were not passing the input_data_format. For images that have 4 channels, this was breaking the processing. Since we already have the input_data_format in these functions, I added it. 

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

- vision models: @qubvel



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34857). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

thanks @andimarafioti  LGTM, I can load the dataset nicely now

@andimarafioti LGTM:  I reran vlmevalkit with the current changes and I can confirm there is no regression

To clarify, here I'm not changing where or if RGB conversion happens.

Yeah, the idea is that we don’t actually need this code because the image will be converted to RGB. However, since maintaining this order is important for quality, it seems fine to me.
Bitnet test fix to avoid using gated  model
Author: MekkCyber
Pull request: yes

# What does this PR do?
Simple fix  to fix the test for bitnet in the CI. Instead of using a gated repo we use a public one.

## Who can review ?
@SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34863). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[CI] Skip EETQ tests while package is broken with latest transformers
Author: BenjaminBossan
Pull request: yes

# What does this PR do?

EETQ tries to import the `shard_checkpoint` function from transformers but the function has been removed. Therefore, trying to use EETQ currently results in an import error. This fix results in EETQ tests being skipped if there is an import error.

The issue has been reported to EETQ:

https://github.com/NetEase-FuXi/EETQ/issues/34

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34854). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
The behavior of QuantizedCache is not consistent with KIVI
Author: ThisisBillhe
Pull request: no
Labels: name: bug

### System Info

transformers              4.45.2

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

As defined in L693-L719 in transformers/cache_utils.py, QuantizedCache will quantize all tokens in the prefill phase (L693-L698) without keeping recent tokens in full-precision (residual_length). Also, during the decoding phase, when self.key_cache[layer_idx].shape[-2] + 1 >= self.residual_length, all tokens are quantized without exception. I think [KIVI](https://github.com/jy-yuan/KIVI) uses a sliding window and always keeps recent tokens of residual_length in full-precision.

### Expected behavior

Always keep the KV cache for "residual_length"  number of tokens in full-precision.

Comments:
Hey @ThisisBillhe !

Yes, the current implementation is not really same as KIVI. As you noted we don''t have have sliding window and we don't use the same quantization as folks from KIVI. KIVI has support for concatenating quantized tensors and for matmuls, while what we do is simply quantize using any open library as backend, and then dequantize it back before concatenating with full precision key/values

The quantized cache in `transformers` is rather inspired by the idea from KIVI paper, so it's not a full copy. At the same time it gives flexibility to switch up `_quantize` and `_dequantize` methods for users to try out different quantization frameworks/methods/params. Improving the cache in terms of latency was an options but unfortunately I am short on bandwidth. Feel free to open a PR to make the cache store key/values in sliding window fashion, if you want to give it a try 😄  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
problem with rag
Author: zahraKane
Pull request: no
Labels: name: rag

name: bug

### System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.44.2
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.24.7
- Safetensors version: 0.4.5
- Accelerate version: 0.34.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.1+cu121 (False)
- Tensorflow version (GPU?): 2.17.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.8.5 (cpu)
- Jax version: 0.4.33
- JaxLib version: 0.4.33
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I tested the rag function with the same dataset 'my_knowledge.csv', everything seemed to work, butThe generated response is not understandable, 
example: Q: What does Moses' rod turn into ?
A:  Powderoo Two stockp upgr Noct vic soy perBleublished *** Offic molecular memElf nu Ore

### Expected behavior

The behavior that i expected is for the rag is to return some coherent responses

Comments:
What is the temperature you are using?


Hey @zahraKane, please provide a reproducible code snippet if you'd like us to help you! thank you

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Drop support for Python 3.8
Author: ydshieh
Pull request: yes

# What does this PR do?

Python 3.8 reached EOL on Oct. 07, 2024

https://devguide.python.org/versions/

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34314). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I will merge once I am able to build the images with python 3.9