noamaanMulla-03/transformers-issues huggingface dataset:
red-ci on main, fix copies
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Make fix copies after #32105 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33356). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix Prefill docs
Author: Rocketknight1
Pull request: yes

One section of the docs still had `continue_last_message`, even though the argument name is actually `continue_final_message`. My bad, this PR fixes it! cc @Tostino 

Also cc @lysandrejik for approval

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33352). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update author for QLorA/PEFT community notebook
Author: daniellok-db
Pull request: yes

# What does this PR do?

So sorry for the trouble! I forgot to fill out the author for #33319. This PR fixes it.

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

cc @LysandreJik 

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Fix exception AttributeError for missing attribute is_zero3 from HfDeepSpeedConfig object
Author: iliacube
Pull request: yes

Fix exception for missing attribute is_zero3 for stage 2 deepspeed.
This issue was observed on Google Colab with runtime GPU A100, when trying to train a Mistral model using deepspeed acceleration.
In the corresponding config json file, stage 2 optimization was specified.


Comments:
@muellerzr please review and merge
Adding NVIDIA Minitron style pruning to llama
Author: gaetanlop
Pull request: yes

This PR adds Minitron style width pruning methodology to llama. The functions added in this PR share a similar idea to _prune_heads (already supported by multiple architectures such as t5 and bert) but also adds the possibility to prune neurons in the FFN layers and the embedding channels as done in the work from NVIDIA Minitron.

I am unsure if HF is interested to support this. If you are interested, I will add the required tests, add some width pruning examples and make the functionalities available to other models such as Mistral.

cc @ArthurZucker


Comments:
Thanks for your feedback @ArthurZucker, I have followed your advice and made a more general PR to adapt pruning to all transformer models in #33377. I think I can close this PR.
Loss Mask to Prevent Weight Updates for Specific Generated Tokens.
Author: naphatkps
Pull request: no
Labels: name: Feature request

### Feature request

We are giving it a text during fine-tuning, but it includes context and a user question with an expected answer. After the model then predicts, during the calculation of loss, it calculates the loss on the whole given input-context, question, and answer. And here I want to apply a loss mask so that it only backpropagates and updates the weights for the loss from the generated answer and not for context and question.

### Motivation

This loss mask feature would ensure that only the loss from the generated answer is backpropagated, excluding the context and question for efficiency in training. It would make the model more focused on generating answers, hence generalizing and avoiding overfitting to the irrelevant parts of the input sequence.

### Your contribution

I am willing to test this feature and report any bugs I encounter during the process.

Comments:
Hey! You can also prepare labels yourself with custom data collators, and set to `-100` the masked part and pad ids. CE loss will ignore the `-100` then 

Indeed, the TRL library for instance has the `DataCollatorForCompletionOnlyLM` class for that purpose: https://huggingface.co/docs/trl/en/sft_trainer#train-on-completions-only.

Thank you so much for your help! I will go through the details of the `DataCollatorForCompletionOnlyLM` and try to apply it into my project.
Fix: Fix build issues with FalconMamba kernels
Author: younesbelkada
Pull request: yes

# What does this PR do?

Currently when a user installs transformers from source and tries to import FalconMambaForCausalLM they will get an error: 
```bash
RuntimeError: Failed to import transformers.models.falcon_mamba.modeling_falcon_mamba because of the following error (look up to see its traceback):
No module named 'transformers.kernels.falcon_mamba'
```
The fix is to add the kernel file in the built transformers package 

cc @ArthurZucker 

Comments:
Feel free to rebase since #33356 should fix your cis

Done @ArthurZucker 

This didn't work for packages that you install with git+... for some reason, I think it's simpler to just move the kernel on the same folder as the modeling file to avoid these issues
Fix quantized cache tests
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Seems like after the last kv-cache clean up, the tests started failing. This PR fixes it, by adding a `quantized` key to possible cache implementation values.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33351). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add visit webpage tool
Author: aymeric-roucher
Pull request: yes
Labels: name: Agents

# What does this PR do?

Adds a tool to visit webpages (once they're found by the DuckDuckGoSearchTool).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33353). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
I am sure I've made a mistake, but confused about the behavior of ``attention_mask'' of the roformer. 
Author: aalennku
Pull request: no
Labels: name: Core: Modeling

name: bug

### System Info

I am confused about the behavior of ``attention_mask'' of the roformer. 
As the document in https://huggingface.co/docs/transformers/en/model_doc/roformer#transformers.RoFormerModel.forward.attention_mask 
said 0 for masked and 1 for not masked. 

but the source code in 

src/transformers/models/roformer/modeling_roformer.py
Line 282 : attention_scores = attention_scores + attention_mask

uses the old fashioned transformer style 0 for unmasked and -inf for masked? 


@ArthurZucker

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

 attention_scores = attention_scores + attention_mask


### Expected behavior

 attention_scores = attention_scores * attention_mask


Comments:
Hey! The attention mask is changed (inverted and extended to 4D) here: https://github.com/huggingface/transformers/blob/0c8021c5d566bcc6897ba3712412414ab47f5e44/src/transformers/models/roformer/modeling_roformer.py#L878-L880

> nded_attention_mask(attention_mask, input_shape)

Thank you!
How to use hugginface for training:  google-t5/t5-base
Author: gg22mm
Pull request: no
Labels: name: Usage

name: Feature request

### Feature request

How to use hugginface for training / 如何使用huggingface来训练：
 https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation

#What is the format and how do I write it? / 这个格式是怎么样的，怎么写呢？
def batch_collator(data):
    print(data)  #?????????????????????????????????????????????   
    return {
        'pixel_values': torch.stack([x for x in pixel_values]), 
        'labels': torch.tensor([x for x in labels]) 
    }

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=batch_collator,//这个需要怎么写?
    train_dataset=dataset['train'],    
)

### Motivation

无

### Your contribution

无


我已经试了可以用： https://www.kaggle.com/code/weililong/google-t5-t5-base 
不知道有没有什么坑

Comments:
You can follow the guide [here](https://github.com/huggingface/transformers/tree/main/examples/pytorch/summarization). Even though it says summarization, you can treat it like translation. 

Summarization goes from article --> summary
Translation goes from start language --> end language
Support Unified Multimodal Model
Author: KevinZeng08
Pull request: no
Labels: name: New model

name: Feature request

name: Multimodal

### Feature request

Hi, I am wondering that can this repository supports the unified multimodal model like Show-o? [https://github.com/showlab/Show-o](https://github.com/showlab/Show-o)

### Motivation

The unified multimodal model may be a trend with multimodality

### Your contribution

trying for integration

Comments:
Hey @KevinZeng08 ! 

Indeed multimodal models are exciting to have! We currently have a PR to integrate Chameleon (https://github.com/huggingface/transformers/pull/32013) with interleaved text-image generation capabilities. Although we don't have work on adding Show-o, there is an issue for people interested in such models (https://github.com/huggingface/transformers/issues/32926). Feel free to comment under the issue and open a PR for Show-o, if you want to give it a try
cannot import name 'Conversation' from 'transformers'
Author: amathews-amd
Pull request: no
Labels: name: Core: Pipeline

name: bug

name: Chat Template

### System Info

transformers tip of tree 

### Who can help?

@amyeroberts

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

pytest tests/ -m "not not_device_test" -p no:cacheprovider

```
______ ERROR collecting tests/pipelines/test_pipelines_conversational.py _______
ImportError while importing test module '/workspace/transformers/tests/pipelines/test_pipelines_conversational.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/conda/envs/py_3.9/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/pipelines/test_pipelines_conversational.py:18: in <module>
    from transformers import (
E   ImportError: cannot import name 'Conversation' from 'transformers' (/workspace/transformers/src/transformers/__init__.py)
```

### Expected behavior

pytest collection succeeds with no error

Comments:
Thanks for reporting! I've opened a PR which should resolve this 

According to that PR 

> Conversational pipeline has been fully deprecated

has it been replaced with anything?

cc @Rocketknight1 on @samgermain's question above!

@samgermain yes, you can pass chats to the `TextGenerationPipeline` instead! See the docs [here](https://huggingface.co/docs/transformers/main/en/chat_templating#is-there-an-automated-pipeline-for-chat).
[`Core tokenization`]  `add_dummy_prefix_space` option to help with latest issues
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Allows users to use `tokenizer.tokenize` controlling the addition of prefix space. Let's also update fast! 

fixes #28622


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28010). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Just wanted to say this would be hugely helpful for us over at https://github.com/probcomp/hfppl !

Likewise the ability to not include an extra SPIECE_UNDERLINE / Llama token 29871 when encoding a word with a space in front ( ` <word>`) would be huge for https://github.com/EleutherAI/lm-evaluation-harness !

will make it for next release I hope! 

Would love to see this as well! Thanks for working on this.

Failing test is unrelated 😉 

@ArthurZucker I know this is old at this point, but this PR seems to introduce a unintended side-effect for Mistral v0.1 tokenizers. I figured out that I can fix it by always initializing with . Not sure if this is on your radar or if it is fixed in newer versions, but would totally appreciate if this can be fixed for the Mistral model series.

Tokens before (4.37.2):

```
SLICED: ['1: 
', '330: A', '28747: :', '28705: ', '28740: 1', '28783: 8', '13: <0x0A>']
```

Tokens after (this PR, `add_prefix_space` not set):

- The "A" vanishes

```
SLICED: ['1: 
', '28747: :', '28705:  ', '28740: 1', '28783: 8', '13: <0x0A>']
```

Tokens after (this PR, `add_prefix_space=True`):

- To be clear, this is the expected default behavior.

```
SLICED: ['1: 
', '330: A', '28747: :', '28705: ', '28740: 1', '28783: 8', '13: <0x0A>']
```

![image](https://github.com/user-attachments/assets/995317df-72e4-4d20-be10-e8e8c9785d5a)

https://www.diffchecker.com/GwA54pMf/
Adjust templates
Author: LysandreJik
Pull request: yes

Adjusting the issue and PR templates reflecting recent and upcoming changes in maintenance:

- Vision: cc @amyeroberts @qubvel 
- Speech: cc @ylacombe and @eustlb while Sanchit is off
- Pipelines: cc @Narsil and @Rocketknight1 (I can put myself as well here if you want @Rocketknight1 to help you shoulder some of the load)

Comments:
@LysandreJik I'm happy to take pipelines! Should we list me for chat / chat templates as well?

Yes, good idea! Will edit

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33384). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Remove repeated prepare_images in processor tests
Author: amyeroberts
Pull request: yes

# What does this PR do?

As per title - moves the logic to the processor tester mixin and its module 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33163). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
The huggingface web site crashed
Author: mokeyish
Pull request: no

<img width="800" src="https://user-images.githubusercontent.com/16131917/159614484-741db4c9-5c60-442a-8df2-efa39869369d.png">

**Set `local_files_only` to be True for reading the local cache first。**

```python
tokenizer = BertTokenizer.from_pretrained(tokenizer_config.model_path, local_files_only=True)
```

# Is there any mirror of this web site?


Comments:
yes, and I couldn't use from_pretrained() in my code

+1

+1

+1

+1

+1

+1

500 Server Error: Internal Server Error for url: https://huggingface.co/bert-base-uncased/resolve/main/config.json

OSError: We couldn't connect to 'https://huggingface.co/' to load this model and it looks like bert-base-uncased is not the path to a directory conaining a config.json file.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

+1

+1

How to prevent Tokenizer/Model from requesting Internet when local cache is available? Force HuggingFace read the local cache first?

@z-bookworm Thanks it worked.

+1

+1

+1

Sorry about that! 
See this tweet for some context on our work on Hub scalability and performance: https://twitter.com/julien_c/status/1496129354023288838

Will close this issue for now, thanks for reporting 🙏
Compile compatibilty for decoder-only models
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Recently we merged a few PRs deprecating old-style cache in all decoder-only models. This PR is a continuation of it, here we verify that all newly deprecated models can support static cache and are compatible with torch.compile. The main change is in RoPE to get rid of dynamic control flow

A few exception that cannot be supported yet: MoE models and some other with dynamic control flow like Phi3 or Chameleon.

Ran `test_generate_compile_fullgraph` and `test_static_cache_matches_dynamic` on all models + ran slow tests on models touched by this PR.

In the next PR I can start deprecating old cache in encoder-decoder models starting from Bart and GPT models

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32617). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Updated with @gante comments and used the new RoPE modeling in all models. Ready for review!

Failing tests are not related

@simonJJJ I added the new RoPE embedding for Qwen2-VL in this PR. Since I changes Qwen2, the changes were automatically propagated with `copy statements`. I remember you had a PR to fix RoPE for FA2 can you check if the current version works as you expect?

@ArthurZucker @gante changed deprecation to v4.46 and added qwen2-VL. Ran the tests again to check everything is okey. Let me know if you have any comments

@zucchini-nlp happy with the changes, feel free to merge! (given that you mentioned that you re-ran the tests 💛 )

Yes, was exactly thinking to rebase main and re-ran tests one more time

Test are passing, including slow. So, merging

Can we update the tracker in https://github.com/huggingface/transformers/issues/28981
Fix import of `FalconMambaForCausalLM`
Author: younesbelkada
Pull request: yes

# What does this PR do?

A cleaner version of https://github.com/huggingface/transformers/pull/33355

cc @ArthurZucker 

Comments:
Thanks a lot, I closed it now

Thanks for the fix! 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33381). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Rest of model init refactors
Author: LysandreJik
Pull request: yes

Builds on top of https://github.com/huggingface/transformers/pull/31329 and applies the rest of the model init refactors.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
VLM: fixes after refactor
Author: zucchini-nlp
Pull request: yes
Labels: name: run-slow

# What does this PR do?

After merging https://github.com/huggingface/transformers/pull/30962 I noticed that some models stopped working in multi-image/image-video setting. Also `cache_position` wasn't being updated accordingly after merging image and text inputs in legacy way. This PR addresses the issue and adds slow tests for that

Also, I think we should use `num_image_tokens` instead of trying to infer it from image size and patch size. The reason is that some image backbones add CLS (CLIP) and other don't (SigLIP). If we try to catch every edge case like this, we'll end up bloating the code, so it's better to let users set the number of tokens needed for one image after encoding with the vision tower. That way we can support different vision towers out-of-the-box

Ran slow tests to test if expanding matches legacy, the new tests and a few older tests. Also changes video llava tests as those weren't formatting the prompt correctly

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32907). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Oke, I will separate these into two PRs and then I have another one, which adds Generation tests in VLMs. That will be a third PR

- Why weren't the breaks caught previously? Was it lack of tests or the test fetcher? If lack of tests, tests should be added which will catch this
Yes, no tests for these cases were added, I added some more. I'll add one more for llava-interleave checkpoint as it uses SigLIP as backbone

- I'm not sure about this switch to num_image_tokens -- can we guarantee this is always fixed? We need to properly handle this change in a non-breaking way
If we didn't change any checkpoint and the latest major release still doesn't have the new processing logic, this shouldn't cause BC errors. WDYT? 

 - Slow tests should be run for all these models
 Yes, will enable in each new PR

To be reviewed and merged after https://github.com/huggingface/transformers/pull/33168

@zucchini-nlp OK - just ping when you want another review! 

@amyeroberts this is ready for review now, the blocking PR is merged. For the tests, the failing ones are SDPA-related which is fixed in https://github.com/huggingface/transformers/pull/32238 (needs a separate review, but not a blocker for current PR). 

The tests are fixed so that we match the runner's output. I verified that the discrepancy appears after updating torch to the latest version and the test was passing in `2.3.0`
fixed Mask2Former image processor segmentation maps handling
Author: maciej-adamiak
Pull request: yes
Labels: name: Vision

Fixes segmentation maps padding size calculation when handling multichannel images using previously inferred `input_data_format`. 

Fixes #33295

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case -> #33295.
- [x] Did you write any new necessary tests?

## Who can review?
Anyone

Comments:
cc @amyeroberts @qubvel