noamaanMulla-03/transformers-issues huggingface dataset:
Add StyleTTS 2 to HF Transformers Pipeline
Author: fakerybakery
Pull request: no
Labels: name: Feature request

### Feature request

Add [StyleTTS](https://github.com/yl4579/StyleTTS2) 2 to HF Transformers Pipeline

### Motivation

Would be great to have an easier way to run STTS2

### Your contribution

I created a [fork](https://github.com/neuralvox/styletts2) with importable scripts

Comments:
cc @sanchit-gandhi @ylacombe 

The model is indeed very performant (see [space](https://huggingface.co/spaces/styletts2/styletts2)) and has a permissive license: one of the packages is GNU licensed. Without this, it can be used with the MIT license

=> on paper it looks like a great candidate for a model addition to Transformers/Diffusers. I wonder if we've missed the boat here with the community interest? WDYT @ylacombe @Vaibhavs10?

Hi, we can replace the GPL phonemizer with gruut to make it MIT

If it would be helpful I can create a demo later with fully MIT licensed components

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Just posting to keep this from going stale

any updates on this?


With the release of Kokoro TTS (https://huggingface.co/hexgrad/Kokoro-82M), a StyleTTS2 finetune, I think it would be amazing to have this model added!
TextIteratorStreamer unable to create generator_kwargs
Author: ivanhe123
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.46.2
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.11.0
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.1.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu121 (True)
- Tensorflow version (GPU?): 2.18.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: Yes
- GPU type: NVIDIA GeForce RTX 4060 Laptop GPU

### Who can help?

@gante

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import TextIteratorStreamer
from threading import Thread
# Load your model and tokenizer
model_id = "./Llama-3.2-3b-instruct"
model = AutoModelForCausalLM.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Define the messages
messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Count to 10."},
]

# Convert messages to inputs
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")

# Create a custom streamer instance with a log file
streamer = TextIteratorStreamer(tokenizer)
generation_kwargs = dict(inputs, streamer=streamer, max_length=256)
thread = Thread(target=model.generate, kwargs=generation_kwargs)
thread.start()
generated_text = ""
for new_text in streamer:
    generated_text += new_text
```

Error:
```cmd
Traceback (most recent call last):
  File "xxxxxxxxxxxx.py", line xx, in <module>
    generation_kwargs = dict(inputs, streamer=streamer, max_length=256)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: dictionary update sequence element #0 has length 46; 2 is required
```

### Expected behavior

Should Run Successfully without any errors.

Comments:
@ivanhe123 hey! The error is stemming not from text streamer, but from inputs. You have to:
`generation_kwargs = dict(input_ids=inputs, streamer=streamer, max_length=256)` to make it work :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Option to Disable Model Caching When Using "pipeline"
Author: FadiAmon
Pull request: no
Labels: name: Feature request

### Feature request

Add a new parameter `disable_caching` to the `pipeline` function to control model caching. This feature would be useful for users who need to load models temporarily without storing them in cache, reducing unnecessary disk usage and improving performance for one-time use cases.

```python
from transformers import pipeline
# Example of disabling caching
pipe = pipeline(task="zero-shot-classification", model="typeform/distilbert-base-uncased-mnli", disable_caching=True)
```

### Motivation

When testing different models temporarily, it's inefficient to cache each model, especially when they will not be reused. This can lead to wasted disk space and potential performance issues and errors for users who frequently experiment with various models.

### Your contribution

I am not very experienced with the library but I am willing to contribute to the initial implementation and testing of this feature if no one wants to work on it.

Comments:
Hi @FadiAmon, I'm not sure we can add this! It's not really possible to load a model without downloading it first, so the files will have to be downloaded somewhere. The only way to reduce cache usage would be to delete the model files **after** the model has finished loading.

If this is a problem for you, you can use the existing `cache_dir` argument to control where models are downloaded. You can then empty this directory as part of your script, or even make it a `TemporaryDirectory()` so it automatically gets cleaned up after the model has been loaded.

@Rocketknight1 Thanks you for the response!
Do you know what is the reason we can't load the model straight from the Hugging Face servers?

Hi @FadiAmon, the way loading works requires the index and all of the safetensors files to be available at once, because tensors can be loaded in a different order than the order of the safetensors shards. We'd have to refactor loading entirely to make it possible to stream weights, and I think it would add a lot of complexity. Also, most users generally want to cache the models rather than download them every single time, so it would require a lot of code for a change that probably wouldn't be used that much.

Alright this makes sense, thanks you! @Rocketknight1
Removed some duplicated code
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
Removed the following duplicated code
https://github.com/huggingface/transformers/blob/15bd3e61f8d3680ca472c9314ad07584d20f7b81/src/transformers/models/paligemma/modeling_paligemma.py#L196
https://github.com/huggingface/transformers/blob/15bd3e61f8d3680ca472c9314ad07584d20f7b81/src/transformers/models/paligemma/modeling_paligemma.py#L199

https://github.com/huggingface/transformers/blob/15bd3e61f8d3680ca472c9314ad07584d20f7b81/src/transformers/tokenization_utils_base.py#L2283-L2289

https://github.com/huggingface/transformers/blob/15bd3e61f8d3680ca472c9314ad07584d20f7b81/src/transformers/tokenization_utils_base.py#L2295-L2301

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35637). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
The Phi model does not have lm_head bias after upgraded to v4.48.0
Author: yuxianq
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.48.0
- Platform: Linux-5.15.0-88-generic-x86_64-with-glibc2.39
- Python version: 3.12.3
- Huggingface_hub version: 0.27.1
- Safetensors version: 0.5.2
- Accelerate version: 1.2.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.6.0a0+ecf3bae40a (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA H200

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The Phi model's lm_head bias disappear after upgraded to v4.48.0, which is incorrect, see https://github.com/huggingface/transformers/blob/e97d7a5be545d123862d09f6c32e8d90b98bbf0c/src/transformers/models/phi/modeling_phi.py#L727

### Expected behavior

Add lm_head bias back to the Phi model.

Comments:
Indeed, thanks for noticing! This will be in the patch!
[`Phi`] bias should be True
Author: ArthurZucker
Pull request: yes

# What does this PR do?
fixes #35625

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35650). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Confirmed that the logits are now good!
Enable different torch dtype in sub models
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/33997. Enables users to use different torch dtypes for each of sub config. For ex load the vision model in full precision and the text model in half precision

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34873). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yeah, this one needs time and I'll come back after the model releases to make sure it works in all cases. Currently it has weird behavior in nested configs where a general text config has an attribute `vision_config`

Conditions to make the dtype dispatch correctly: use `_from_config` to init the submodule inside a bigger model as `ChameleonVQVAE._from_config`. That way the default dtype is set before initilizing the weights on each submodule, otherwise they are not set

The current design support setting dtype via `dict` when loading or directly via changing the config. The dict should have the same keys as sub-configs and can have an empty string key which will be the main dtype. Main dtype is used for any module that is not part of sub-config, for ex `mm_projector`

Added more tests and verified it works when `keep_in_fp32_modules` is indicated. For the case when model's forcefully limit dtype of certain classes, I haven't found an actual model that does it so not sure how to test that

Not breaking anymore, will need at least one approve to merge :)

cool, since release is done will rebase and merge
Will Qwen2VL support sequence classification head in the future?
Author: cv-nlp
Pull request: no
Labels: name: Feature request

### Feature request
Will Qwen2VL support sequence classification head in the future?

Comments:
Hey @cv-nlp !

This issue is loosely related to https://github.com/huggingface/transformers/pull/34086. Tl;DR; we are happy to support a new class for sequence classification as long as there are trained checkpoints for that task. Do you know if qwen2-vl has trained models for sequence classification on the hub? 

Feel free to open a PR if you find models :)
[`Compile`] Only test compiling model forward pass
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Instead of testing the full generate compile. 

Comments:
cc @zucchini-nlp as we chatted on friday, let's not spend more energy on end-to-end compile

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35658). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add dinov2 with registers attempt 2
Author: BernardZach
Pull request: yes
Labels: name: New model

name: Vision

name: run-slow



# What does this PR do?

This PR is a re-follow up and rebase-ing of #32905  and it seems to be working well from the work done by @NielsRogge.
my first attempt (which is now closed) #34788 was too far gone and I thought it better just to start fresh now that I am more familiar with transformers. 

Thank you all for your patience.

---

Relevent reviewers:
@ArthurZucker



Comments:
Add-helium
Author: ArthurZucker
Pull request: yes
Labels: name: New model

# What does this PR do?
Add the helium model from Kyutai

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35669). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Moonshine logits processor
Author: eustlb
Pull request: yes

# What does this PR do?

This PR introduces a new logits processor that enables per-batch-index maximum lengths.

This feature is particularly useful for audio encoder-decoder models that rely on `generate` and use a heuristic based on a `token_limit_factor`. In such cases, the maximum length can be inferred from the input length, given a token/sec maximum. This approach helps reduce model hallucinations.

The logits processor is designed to be passed via the `logits_processor` argument of the `generate` method. Additionally, this PR leverages the logits processor in the Moonshine model, where per-batch-index maximum lengths are inferred from the provided inputs.

Comments:
Fix multi-gpu loss
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the label's device after #34191

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35395). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thank you!
Fix `zero_shot_image_classification` documentation guide link in SigLIP
Author: aretrace
Pull request: yes

# What does this PR do?
Fixes the computer vision zero-shot image classification guide link in the SigLIP docs 

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?
@stevhliu

Comments:
Fix : adding einops lib in the CI docker for some bitsandbytes tests
Author: MekkCyber
Pull request: yes

# What does this PR do?
Adds the einops library in docker for some `bitsandbytes` tests
<img width="1040" alt="Screenshot 2025-01-13 at 10 58 31" src="https://github.com/user-attachments/assets/fefda907-0665-4f16-9bc6-8c79f899e3ed" />

https://github.com/huggingface/transformers/actions/runs/12729761010/job/35482083926
## Who can review ?
@ydshieh 

Comments:
Hi! I think the correct docker file should be

> docker/transformers-quantization-latest-gpu/Dockerfile

but let me know if I am wrong.

(and it would be better to (also) share a job run page link too - if you find them on those run page of course)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35652). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @ydshieh, it's in the quantization part of the model CI not the quantization CI I just shared the job link
Special token ids are not longer typed properly in 4.47.0
Author: chanind
Pull request: no
Labels: name: bug

### System Info

tranformers 4.47.0, python 3.11

### Who can help?

@ArthurZucker (I think)

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`SpecialTokensMixin` in `tokenization_utils_base` returns type of `list[str | list[str] | Unknown] | Unknown` for `bos_token_id` and other special token IDs, whereas previously it was correctly `int | None`. This is a regression caused by the following commit, where all special token types were (likely unintentionally) deleted: https://github.com/huggingface/transformers/pull/34461/files#diff-85b29486a884f445b1014a26fecfb189141f2e6b09f4ae701ee758a754fddcc1 

### Expected behavior

special token ids on tokenizers should keep returning type `id | None`

Comments:
cc @zucchini-nlp let's try to fix this!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix : HQQ config when hqq not available
Author: MekkCyber
Pull request: yes

# What does this PR do?
Fix hqq config when hqq not available

## Who can review ?
@SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35655). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix expected output for ggml test
Author: MekkCyber
Pull request: yes

# What does this PR do?
Fix the expected output for ggml test, tested on H100 and A100
The CI error : https://github.com/huggingface/transformers/actions/runs/12759985366/job/35564906527

## Who can review ?
@SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35686). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix : add require_read_token for gemma2 gated model
Author: MekkCyber
Pull request: yes

# What does this PR do?
Adds `require_read_token` for gemma2 gated model in `test_ggml`

## Who can review ?
@SunMarc 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35687). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
test_custom_4d_attention_mask skip with sliding window attn
Author: poedator
Pull request: yes

This is a follow-up to #30348 
`tests/models/qwen2/test_modeling_qwen2.py::Qwen2ModelTest::test_custom_4d_attention_mask` was failing because of use of sliding window attention. 
While this sliding window attention does not preclude use of custom 4D masks, it requires modification of the original `test_custom_4d_attention_mask`.

Summoning @ArthurZucker @gante to review

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30833). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
4D mask documentation updates
Author: poedator
Pull request: yes

following https://github.com/huggingface/transformers/pull/27539 this PR adds updates to transformers documentation to reflect possibility of utilizing 4D masks.

Plan:
- add updates for Llama model docstring(s)
- identify other models that can use 4D masks in present form (which requires ability to accept custom `position_ids` argument) and updating their docstrings. Classes that need updates:
    -  Falcon Model
    - [TODO identify more]
- update code comments that may need corrections, like cases where the mask may be either 2D or 4D now. one example is based on [this comment](https://github.com/huggingface/transformers/pull/27539#issuecomment-1863285474) by @shentianxiao

Update 20.12.2023:
to find out which models require docstring changes, I scanned all model classes in transformers insing inspect.
- excluded tf and jax classes
- excluded models without `position_ids` argument in `.forward()` - can't use 4D mask effectively
- excluded models that do not use `_prepare_4d_attention_mask` method - need different code change to use 4D mask
- excluded multi-modal models (clip, clvp, vit, bark, git)

what is left is LlamaModel, FalconModel and XGLMModel.

cc @ArthurZucker 

Comments:
Feel free to ping me for a review whenever this is ready 🤗 

> Feel free to ping me for a review whenever this is ready 🤗

@ArthurZucker , I only identified 3 applicable model classes and made changes. Please check my logic in classes selection in my big first message above.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Has something like this or equivalent been merged? I don't see much documentation for 4D masks but would find useful!

Yep sorry, it's supported for some models. https://github.com/huggingface/transformers/blob/18527bd029316037dd174c85b1f9b55288b50c6c/src/transformers/models/bloom/modeling_bloom.py#L65 doc is a bit scarce, feel free to open a pr to add this!