noamaanMulla-03/transformers-issues huggingface dataset:
Guard XLA version imports
Author: muellerzr
Pull request: yes

# What does this PR do?

As reported by @michaelbenayoun, spmd and runtime imports are only available after torch_xla 2.2.

This PR adds some guards around those imports


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30167). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Wav2Vec2CTCTokenizer does not add blank token between repeated letters
Author: nikvaessen
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.2
- Platform: Linux-5.19.11-arch1-1-x86_64-with-glibc2.36
- Python version: 3.10.0
- Huggingface_hub version: 0.10.0
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


### Who can help?

@patrickvonplaten, @anton-l

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

When encoding/decoding with `Wav2Vec2CTCTokenizer`, I expect blank tokens between repeated letters. If they are left out, a model will never be able to learn to correctly spell any word with repeated letters.

```python3
import transformers

tokenizer = transformers.Wav2Vec2CTCTokenizer.from_pretrained(
    "facebook/wav2vec2-base-960h"
)

transcription = "ee".upper()

input_ids = tokenizer(text=transcription)['input_ids']
print(input_ids)
print(tokenizer.decode(input_ids))
```

Output:
```
> [5, 5]
> E
```

### Expected behavior

Output should be 
```
> [5, 0, 5]
> EE
```

Comments:
Hi @nikvaessen I have the same issue -- did you find a solution to this? I'm currently implementing my own logic to insert the blanks.

Yes, you need to use `tokenizer(text_target=transcription)['input_ids']` instead of `tokenizer(text=transcription)['input_ids']`
[Processor classes] Update docs
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR updates the docs of 13 multimodal processor classes, which state that `images` should be in the channels-first format, whereas both channels-first and channels-last are supported, as pointed out in https://github.com/huggingface/transformers/pull/29586#discussion_r1523430971.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29698). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Allow smart context to use accelerator
Author: muellerzr
Pull request: yes

# What does this PR do?

I believe this fixes the issue in https://github.com/huggingface/transformers/issues/28512, we weren't picking up the `Accelerator`'s context manager if it had one.

Fixes https://github.com/huggingface/transformers/issues/28512


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@amyeroberts 

Comments:
I did verify there's no computational difference in double auto-casting (just in case). (double meaning one from the model, the other from the context manager)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30163). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This may be a more complicated fix needed, marking as a draft for a bit

Bringing it back, I think it's fine. Waiting for the user to report back how they could tell of this issue first and then will write a test based on that exact scenario (if not I'll just write a new one)
ENH: [`CI`] Add new workflow to run slow tests of important models on push main if they are modified
Author: younesbelkada
Pull request: yes

As per title, WIP ..

cc @ydshieh 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29235). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The PR is ready for review! IMO we can merge it and make a follow up PR to fix the failing tests 
The results can be seen in #transforers-ci-push-important-models channel

This PR is ready for review! 

Testing on all supported models in [c99481f](https://github.com/huggingface/transformers/pull/29235/commits/c99481f24240f0b3121493eafa91589c4722c880) before merging

ok everything is failing, will fix everything in this PR and merge it

This PR is ready to be shipped once #29805 is merged! 

The workflow is ready ! to add whisper / llava it will require some work, I 'll do that in a follow up PR!

Left 2 nits, otherwise LGTM.
Fix defaults + correct error in documentation for Mixtral configuration
Author: kalomaze
Pull request: yes

# What does this PR do?
- The default value for the `max_position_embeddings` was erroneously set to 4096 * 32. This has been corrected to `32768`
- Mixtral does not use Sliding Window Attention, it is set to null in the official config.json. So, the notice about the model using SWA has been removed.




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Inconsistent hidden_states in inference
Author: xxyyzztlb
Pull request: no

I want to get the tensor of the last layer in the inference stage. The batch_size is 1. I use the following two methods:
1:
`model = LlamaForCausalLM.from_pretrained(args.base_model,
            torch_dtype=torch.bfloat16,
            low_cpu_mem_usage=True,
            device_map=device_map,
        )`

`output = model.module.generate(input_ids=inputs["input_ids"],
          attention_mask=inputs["attention_mask"],
          max_new_tokens=10,
          prefix_allowed_tokens_fn=prefix_allowed_tokens,
          num_beams=20,
          num_return_sequences=20,
          output_scores=True,
          return_dict_in_generate=True,
          early_stopping=True,
          output_hidden_states=True
      )`

`last_output = output['hidden_states'][0][-1][0, -1, :]`




2:
`model = LlamaForCausalLM.from_pretrained(args.base_model,
            torch_dtype=torch.bfloat16,
            low_cpu_mem_usage=True,
            device_map=device_map,
        )`

`outputs = model(input_ids=inputs["input_ids"],
        attention_mask=inputs["attention_mask"], output_hidden_states=True)`

`last_output = outputs.hidden_states[-1][0, -1, :]`

I found that the tensor values ​​obtained are different. I'd like to know what are the potential reasons.

Comments:
Hey @xxyyzztlb. Please take a look at [this comment](https://github.com/huggingface/transformers/issues/25420#issuecomment-1775317535). TL;DR: we cannot guarantee exactly same values for hidden states, due to numerical precision errors. Given that your code snippet is using `float16`, the error might be even higher than `float32`

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add ViTMatte model
Author: amyeroberts
Pull request: no
Labels: name: New model

name: Vision

### Model description

ViTMatte is a recently released model for alpha matting on images i.e. background removal. 

The model accepts an input image and trimap (manually labelled grayscale image outlining the rough border of the foreground object) and predicts the alpha mate for each pixel. 

It introduces a series of small adaptations to the ViT architecture - selective global attention + window attention; adding convolutional blocks between transformers blocks - to reduce computational complexity and enhancing the high-frequency information passed through the network. 

At the time of publishing, ViTMatte showed SOTA performance on Distinctions-646 and strong performance (> Mattformer) on Composition-1K.  

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Github: https://github.com/hustvl/ViTMatte
Paper: https://arxiv.org/pdf/2305.15272.pdf
Demo: https://colab.research.google.com/drive/1Dc2qoJueNZQyrTU19sIcrPyRDmvuMTF3?usp=sharing

Comments:
This can be closed now thanks to #25843
ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils' (~/lib/python3.10/site-packages/transformers/utils/__init__.py)
Author: Rajmehta123
Pull request: no

### System Info

transformers: 4.35.0
python: 3.10.13
Platform: Linux

### Who can help?

@Narsil @ArthurZucker @SunMarc 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch
model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-6B-200K", trust_remote_code=True,torch_dtype=torch.bfloat16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("01-ai/Yi-6B-200K",trust_remote_code=True)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)

This works fine with 4.34.1 transformers verion

### Expected behavior

The model gets loaded correctly

BUT I got this error: 

ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils' (~/lib/python3.10/site-packages/transformers/utils/__init__.py)

This works fine with 4.34.1 transformers verion


Comments:
Hi @Rajmehta123 ! Thanks for reporting this issue. This happens because in the latest version of transformers, we removed `is_flash_attn_available()` in favor of `is_flash_attn_2_available()`. See related [PR](https://github.com/huggingface/transformers/pull/26785). I guess the remote [code](https://huggingface.co/01-ai/Yi-6B-200K/blob/96eff7e8c7871e95d59d13497923800e084ea7b0/modeling_yi.py#L21) that you are executing is using `is_flash_attn_available`, hence the error. I've opened an PR to put back the function and make it go through a deprecation cycle ! 

Hi @SunMarc , I have an error as well but its on the flip side

from transformers import pipeline
classifier = pipeline('summarization')

RuntimeError: Failed to import transformers.models.bart.modeling_bart because of the following error (look up to see its traceback):
cannot import name 'is_flash_attn_2_available' from 'transformers.utils'

Transformers: Version: 4.35.0
Python: 3.11.4

Hi @regineshalom, i'm unable to reproduce your error on my local setup. If you can reproduce the error in a colab and send it to me, that would be great. 

ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils'
答：`pip install transformers==4.34.1`，transformers的版本必须是4.34，不能是4.31、4.32、4.33

![image](https://github.com/OrionStarAI/Orion/assets/32784059/49b8b467-072a-4661-918b-7a68d4673199)

I got the same `cannot import name 'is_flash_attn_2_available' from 'transformers.utils`
What version i downgrade to to get it working? thanks

I think upgrading would be better 😉 

> I got the same `cannot import name 'is_flash_attn_2_available' from 'transformers.utils` What version i downgrade to to get it working? thanks

```
# Install the specific version using pip
pip install transformers==4.34.1
```

@curlup    It work fine to me.
HfDeepSpeedConfig + ZeRO3 init accuracy bug!
Author: hijkzzz
Pull request: no

### System Info

see https://github.com/microsoft/DeepSpeed/issues/4932

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

https://github.com/microsoft/DeepSpeed/issues/4932

### Expected behavior

https://github.com/microsoft/DeepSpeed/issues/4932

Comments:
https://github.com/microsoft/DeepSpeed/issues/4932

Hi @hijkzzz, thanks for raising an issue! 

cc @pacman100 @ArthurZucker cf [this comment](https://github.com/microsoft/DeepSpeed/issues/4932#issuecomment-1902177942)

Also cc @gante as it seems related to ROPE! 

To prevent duplication, let's focus all discussion on https://github.com/huggingface/transformers/issues/28685 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[Mamba] Possible Issue in beam search with Mamba HF models
Author: Mirmu
Pull request: no
Labels: name: Feature request

### System Info

nightly build transformers version

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda:0"
tokenizer = AutoTokenizer.from_pretrained("state-spaces/mamba-130m-hf")
model = AutoModelForCausalLM.from_pretrained("state-spaces/mamba-130m-hf").to(device)

input_prompts = ["Question: Who is the lead singer of Coldplay? Answer:",]
prompted_encoded = tokenizer(input_prompts[0:1], return_tensors="pt", padding=True).to(device)['input_ids']

with torch.inference_mode():
        generated_tokens = model.generate(input_ids=prompted_encoded, max_length=60, use_cache=True, num_beams=5)
        decoded_tokens = tokenizer.batch_decode(generated_tokens)
        print("Generation finished.")
        print(decoded_tokens)
```

### Expected behavior

We are observing a degradation of the mamba model generated output as `num_beams` parameter goes up. 

Example with num_beams=1
```
Generation finished.
['Question: Who is the lead singer of Coldplay? Answer: The lead singer of Coldplay is the lead singer of the band Coldplay.\n\nThe lead singer of Coldplay is the lead singer of the band Coldplay. The lead singer of Coldplay is the lead singer of the band']
```

Example with num_beams=100:
```
Generation finished.
['Question: Who is the lead singer of Coldplay? Answer: Coldplay\n\n\n Cold isfield\n\n\nQuestion:\nColdplayplayfield\n\n\n:\n\n:\n\n:\n\n:\n\n:\n\n:\n\n:\n\n:\n\n:\n']
```

Comments:
I'll mark this as a feature request as I am not sure the original code had this either, and yes, beam generation donc look really  good 😅 but that is probably because the logic of the state is different than the cache that is reordered. I'll see what I can do! 

Thanks for the pointer, Arthur. The issue indeed seems coming from the re-ordering logic, which I could fix easily (handling is similar to the past_key_values one), with the following method:
```
    def reorder_cache(self, beam_idx):
        for layer_idx in self.conv_states:
            device = self.conv_states[layer_idx].device
            self.conv_states[layer_idx] = self.conv_states[layer_idx].index_select(0, beam_idx.to(device))
            device = self.ssm_states[layer_idx].device
            self.ssm_states[layer_idx] = self.ssm_states[layer_idx].index_select(0, beam_idx.to(device))
```


Feel free to open a PR for the fix!

Hi @Mirmu @ArthurZucker ! Is there any PR finally for this? thanks!
fix typo
Author: mdeff
Pull request: yes

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
fix doctest
Author: molbap
Pull request: yes

# What does this PR do?

Should fix a failing doctest captured by `HfDocTestParser`.

@ydshieh 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30215). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix `RecurrentGemmaIntegrationTest.test_2b_sample`
Author: ydshieh
Pull request: yes

# What does this PR do?

Update the expect values (it's likely @ArthurZucker is using DGX or A100 when adding the tests)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30222). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix/Update for doctest
Author: ydshieh
Pull request: yes

# What does this PR do?

Doctest is failing for some time. #30210 make it run again. This PR fixes a few failures observed after running doctest  enabled by #30210 .

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30216). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fixed config.json download to go to user-supplied cache directory
Author: ulatekh
Pull request: yes

Before, config.json would end up in the default cache directory. Now everything but version.txt is placed properly, and since it's downloaded when pipeline is imported, fixing that will be more complex.

I ran "pytest --picked"; it generated 3 warnings, none of which seemed to have anything to do with my change.

@Narsil since this is related to pipeline.

Comments:
Google Colab: ValueError: Connection error, and we cannot find the requested files in the cached path.
Author: keyuchen21
Pull request: no

### System Info

- `transformers` version: 4.36.2
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.1+cu121 (False)
- Tensorflow version (GPU?): 2.15.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.8.2 (cpu)
- Jax version: 0.4.23
- JaxLib version: 0.4.23
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

you can use this colab: https://colab.research.google.com/drive/1zn42joYvWmwksPrUlbxRzwxEPJeJuU3n?usp=sharing

!pip install adapters
from adapters import AutoAdapterModel

# 1.
model = AutoAdapterModel.from_pretrained("FacebookAI/roberta-base")
# 2.
adapter_name = model.load_adapter("AdapterHub/roberta-base-pf-imdb")
# 3.
model.active_adapters = adapter_name
# or model.train_adapter(adapter_name)

### Expected behavior

<img width="1383" alt="image" src="https://github.com/huggingface/transformers/assets/54015474/31306c6a-9daa-4f35-946f-547e60e5d3eb">


but somehow it works for other models?
<img width="901" alt="image" src="https://github.com/huggingface/transformers/assets/54015474/85fb7eb9-fe76-4757-994b-a3198b0243a0">

but the samples from huggingface official documents doesn't work
<img width="1844" alt="image" src="https://github.com/huggingface/transformers/assets/54015474/15fc2f69-1bca-4eb2-af6f-b9aee739065b">



Comments:
Hi @km5ar, thanks for raising this issue! 

I'm able to replicate this issue. From the traceback, the error seems to be triggered within the `adapters` library, rather than `transformers`. I'd suggest raising an issue in [that repo](https://github.com/adapter-hub/adapters)