noamaanMulla-03/transformers-issues huggingface dataset:
Generate: replace breaks by a loop condition
Author: gante
Pull request: yes

# What does this PR do?

Pulled from the `torch.compile(..., fullgraph=True)` draft PR: https://github.com/huggingface/transformers/pull/29374

It replaces the `breaks` that exit the endless generation loop with an equivalent function that returns `False` when it should stop generating, while preserving ZeRO stage 3 support. It is not only an improvement in terms of code reuse, but also a hard requirement to enable  `torch.compile(..., fullgraph=True)`: `break` and data-dependent control flow is not supported.



Comments:
FYI @zucchini-nlp (the stopping criteria solution did not preserve ZeRO stage 3 support)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29662). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[FIX] Fix speech2test modeling tests
Author: ylacombe
Pull request: yes

# What does this PR do?

Fixes the modeling tests of speech2text.
I've also run the slow tests of course!

cc @ydshieh and @amyeroberts 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29672). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Revert "Fix wrong condition used in `filter_models`"
Author: ydshieh
Pull request: yes

Reverts huggingface/transformers#29673

Sorry, I am not aware we have issue with 

> /bin/bash: line 25: /usr/bin/jq: Argument list too long

because we have too many models now!

See the file [test_preparation/generated_config.txt](https://app.circleci.com/pipelines/github/huggingface/transformers/87251/workflows/d1af48ac-13ee-425d-b1ab-f7e4f4d00003/jobs/1133645/artifacts)

We have to rethink how we can do, but let's revert the PR asap so future merge events won't face such issues.



Comments:
cc @gante 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29682). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sure. Thank you for approving!

track in #29683
Small Bug in Encoder Implementation
Author: manoja328
Pull request: no

### System Info

any version , latest version

### Who can help?

@ArthurZucker   @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/blob/a1afec9e1759b0fdb256d41d429161cc15ecf500/src/transformers/models/mobilebert/modeling_mobilebert.py#L585 

head_mask[i] will not exists if head_mask supplied is None. And this will give an error. 

Solution: in the inner loop  
`head_mask = head_mask[i] if head_mask is not None else None`

### Expected behavior

Solution: in the inner loop  
`head_mask = head_mask[i] if head_mask is not None else None`

Comments:
Got and error when tried to pass embeddings 

output = encoder(embeddings)


Hi @manoja328 
Thanks for the issue! This is I think expected as this class is not meant to be used as a standalone class, `head_mask` is automatically created as a list of `None` here: https://github.com/huggingface/transformers/blob/1c31b7aa3bb4e7ef24c77596d2a76f45a770159f/src/transformers/modeling_tf_utils.py#L1219
but I agree we could make it more userfriendly, one way could be to initialize `head_mask` in the encoder to be  `[None for _ in range(len(self.layers))]` to preserve `head_mask[i]` and keep a consistency across expected arguments.  

Yup, that should be sufficient. I encountered this while I was trying to compute attribution of logits with respect the input embeddings using jacobians. So there first i computed embeddings for the sequence then if I  pass the embeddings to the encoder to get the logits, which is when I encountered this error.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ValueError is raised when `num_return_sequences > num_beams` even if `do_sample` is True
Author: JohnnieDavidov
Pull request: no

### System Info

In [configuration_utils.py](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/configuration_utils.py#L506) an error is raised if `num_return_sequences > num_beams`, but I think that this error should be raised only when do_sample is False. This is because in [utils.py](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L1519) if the generation mode is set to sample, then we expand the input to create multiple generations, and do not rely on the beam size.
If this makes sense, I can open a PR that addresses this issue by adding a condition that checks if we are in a sample generation mode.

@gante I would appreciate your input on this. This validation was added in [this PR](https://github.com/huggingface/transformers/commit/5bd8c011bb24201148c1a2ba753ffd3f0822f004).

Thanks


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

[this line](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/configuration_utils.py#L506)
 and I get this Error:
 `num_return_sequences` (30) has to be smaller or equal to `num_beams` (4).

### Expected behavior

To find out if there's a bug in what I'm thinking.

Comments:
I personally don't think this is a bug but rather a correct behavior because this error is only triggered when the param `num_beams` is set, which means the user either opt `beam search` or `beam sample`. Both the two sampling methods only keep `num_beams` candidate during searching so they really can not return more than `num_beams` generations. 

(N.B. beam sample != sample multiple times)

If the user wants to get multiple generations from random sampling, the `num_beams` param should be removed. 

@JohnnieDavidov 👋  @Saibo-creator wrote the correct answer :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
token-classification
Author: AdrianRemo14
Pull request: no

### System Info

everything necessary for the operation of the program

### Who can help?

@ArthurZucker  @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

i use this token-classification https://github.com/huggingface/transformers/tree/main/examples/pytorch/token-classification

### Expected behavior

![warning](https://github.com/huggingface/transformers/assets/159118361/e9d16cad-c070-466d-b7f3-eb156533c1eb)

i use this format for json:
{"text": ["Ă„ÂŚie", "sĂÂş", "to", "tenisky", "Peter", "naozaj", "sĂÂş", "tvoje", "Tak", "si", "ich", "hneĂ„ÂŹ", "uloĂ…Âľ"], "labels": ["O", "O", "O", "O", "QUESTION", "O", "COMMA", "O", "O", "O", "QUESTION", "O", "O", "O", "O", "O", "EXCLAMATION"]}
{"text": ["Pri", "rieke", "stojĂÂ­", "mlyn", "Z", "chaty", "sa", "dymĂÂ­", "Dedo", "Ă„ÂŤĂÂ­ta", "noviny"], "labels": ["O", "O", "COMMA", "O", "O", "PERIOD", "O", "O", "O", "O", "PERIOD", "O", "COMMA", "O", "O", "PERIOD"]}

I want to ask why I get a warning that my tag is not in the NE tag. I want to train slovakBERT on my own data. Can you advise me what I'm doing wrong or what I need to fix?

Comments:
Hey 🤗  thanks for opening an issue! We try to keep the github issues for bugs/feature requests! The example scrips should be adapted to each's use case, and I suppose that is what you did. As such I would recommend 2 things:
-  ask your question on the [forum](https://discuss.huggingface.co/) 
- track the warning's source to check what is happening. (Question is probably missing from the vocab NE?) 
 

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add predict_proba method for Autoformer/Informer
Author: benHeid
Pull request: no
Labels: name: Feature request

### Feature request

Currently, the autoformer and informer does only provide sampled outputs from a probability distribution for future values. However, it would be nice if there would be the possibility to provide the forecasted distribution to the user.

### Motivation

I am currently trying to develop an adapter in sktime that enables the integration of the time series models from the transformers library (https://github.com/sktime/sktime/issues/5790). Since sktime has a `predict_proba` method, I would like to translate the transformers probability distribution into sktime probability distributions. 

### Your contribution

I think there are at least three solutions:
1. Add a new method, e.g., `predict_distribution`, which is returning the distribution object or its parameters.
2. Add a parameter to the `generate` function that controls if the distribution or its parameters are returned. 
3. Always returns the distributions or it's parameters together with the sampled `sequence` in `generate`.

If you are preferring any of these three solutions, I am happy to implement it :) 

Comments:
WDYT @kashif ?

right so i think the most useful would be that we return in the generated output datasclass the param. of the distribution for each time step... this way one can reconstruct the distribution... generate of course internally will need samples for the autoregressive generation to feed the next time point... so i think 3 would be the easiest.

Would the seq of return param be fine for you @benHeid ?

so we can extend `SampleTSPredictionOutput` to also contain optional parameters ... the one caveat is that different parametric distribution heads have different numbers of parameters, gaussian is 2, student-t is 3 and neg. bin is also 3 per time step

Returning a sequence of params would be fine for me. This would enable me than to just pass these parameters to the sktime distribution objects. 

I try to create a corresponding PR this weekend :)
Will deep StateSpace models add to this library?
Author: ghosthamlet
Pull request: no
Labels: name: Feature request

### Feature request

Will huggingface add deep StateSpace models to transformers library or create a new repo like Diffusers?

### Motivation

Deep StateSpace models may become the next big thing.

### Your contribution

No

Comments:
https://github.com/huggingface/transformers/pull/28094
Fix links in README translations for Llama2 Models
Author: JuanFKurucz
Pull request: yes

# What does this PR do?

Following the fixes on the #26640 PR, we should fix those same links in the translations for each README file.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Documentation: @stevhliu and @MKhalusova


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28910). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Potential error for llava generation
Author: ByungKwanLee
Pull request: no

https://github.com/huggingface/transformers/blame/abf8f54a019ce14b5eaffa68c6dd883be13fe66e/src/transformers/models/llava/modeling_llava.py#L412

This line has potentially critical bug when they do not exist such that input_ids has -100 index unexpectedly.

Therefore I edit that part

![image](https://github.com/huggingface/transformers/assets/50401429/1be763f5-e9fb-4c0b-bfab-f097237f7d9e)




Comments:
In transformers the input ids cannot be negative 😉  

However, I have experienced the negative input ids for the batch-generating text

Can you share a reproducer? 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
add regnet chinese doc
Author: a-strong-python
Pull request: yes

Add Chinese reference documents to the regnet model

Comments:
@vanpelt 

cc @stevhliu 

> Thanks for translating this! 🤗 
> 
> Everything after the `## Resources` section can be left untranslated. These classes/methods are automatically documented with our special doc-builder syntax, [`[[autodoc]]`](https://github.com/huggingface/transformers/blob/4830f2696575988faee4af78b6049b62a750ecd4/docs/source/en/model_doc/regnet.md?plain=1#L47).

I know that the `## Resources` section can automatically generate documents, but in this case most of the documentation comments will still be in English and cannot be translated into Chinese.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
CI / generate: batch size computation compatible with all models
Author: gante
Pull request: yes

# What does this PR do?

Fixes the CI issues in musicgen.

The following tests were run locally (with the expected tests passing):
1. `py.test tests/models/musicgen/test_modeling_musicgen.py -vv`
2. `RUN_SLOW=1 py.test tests/models/llama/test_modeling_llama.py -vv` [because the failing changes were initially made for this model]

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29671). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add MusicGen Melody
Author: ylacombe
Pull request: yes

# What does this PR do?

MusicGen Melody was released at the same time than the "o.g" MusicGen that has already been integrated to [`transformers`](https://huggingface.co/docs/transformers/v4.37.2/en/model_doc/musicgen#overview).

Contrarily to the already integrated model, you can condition the generation with an audio prompt (instead of continuation of the audio prompt).

**Main conceptual difference**-> we no longer use cross-attention to condition the generation with the text/audio prompt, but instead we concatenate the text/audio prompt to the decoder hidden states.

This makes the model a bit simpler, since it's no longer a "proper" encoder-decoder architecture but a decoder-only that can be conditioned (a bit like Fuyu).

Note that there are 3 key "modalities":
-> the prompt text that is passed through a text encoder model.
-> the audio prompt that is processed by the feature extractor to give a chromagram.
-> the musicgen decoder, that generate [Encodec](https://huggingface.co/docs/transformers/v4.37.2/en/model_doc/encodec#overview) codes. 

Why is this model interesting?
1. Audio prompting instead of audio generation gives really interesting generation
2. Musicgen is a decoder-only model, and is difficult to train using the original library. I ideally plan to add training capabilities to the model.
 

cc @sanchit-gandhi 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28819). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hey @sanchit-gandhi, thanks for the really relevant suggestions! I've addressed most of them! Could you give it another tour before asking a core maintainer's opinion ?

BTW, I don't know yet how to efficiently initialize `audio_enc_to_dec_proj` that is part of `MusicgenMelodyForConditionalGeneration`, any opinion on this ? 

It's mostly corrected, asking for @ArthurZucker review now !
Also cc @gante, could you take a quick look at the custom generation methods ? It's mostly the same as Musicgen but in a decoder-only settings!

@ylacombe could you make sure CI's are green! 🤗 

Hey @ArthurZucker, should be okay now, there is still the documentation tests that won't pass yet, but it's related to #28905 !

Thanks, I'll have to review tomorrow 😉 

@ylacombe at a quick glance, the `generate` method looks the same as in MusicGen, which has been previously approved. Are there differences that you'd like me to review? 🤗 

>  I'd like to request a follow up PR where a lot of this is abstracted out into smaller, more modular methods.

@amyeroberts 100% agreed! However, I believe the ball is mostly on the `generate` side -- it should be made more flexible, such that enabling models like MusicGen becomes a < 100-line task.

The MusicGen PR had the exact same pattern :)
FIX [`bnb`] Make `unexpected_keys` optional
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes: https://github.com/huggingface/transformers/issues/29193 

The code path to: https://github.com/huggingface/transformers/blob/831bc25d8fdb85768402f772cf65cc3d7872b211/src/transformers/modeling_utils.py#L3930 is accessed only when users use FSDP. The fix is to simply make `unexpected_keys` optional so that it will not throw the error shown in the issue




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29420). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Support resuming of deepspeed + Lora + offloading
Author: thepowerfuldeez
Pull request: yes

This PR is a upstream version of @kazemf78 PR to support resuming of Lora training when using deepspeed.
Without setting `load_module_strict=False` as a default, checkpoint is not loaded due to Lora not containing all weights, throwing an error `deepspeed resume Error(s) in loading state_dict for PeftModelForCausalLM`

Related discussion: https://github.com/huggingface/peft/issues/746

Comments:
cc @pacman100 @younesbelkada 

Could you please provide any updates on this PR?

Sure @thepowerfuldeez ! 
@pacman100 is currently working on fixing issues with repsect to deepspeed and providing working scripts that you can run out of the box: https://github.com/huggingface/peft/pull/1489 we'll review this PR asap with sourab!

Hello, this has been already fixed in https://github.com/huggingface/transformers/pull/28746. I ran experiments today and can confirm resuming training when using PEFT+DeepSpeed works

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I'd like to bump this I'm running into the this issue on my project and it's causing significant delays. Does this branch solve the issue but is not tested enough for merging to main or is a solution still to be found?

Hi @ambroser53 ! I haven’t tested this branch on an upstream, but it should work.

> I'd like to bump this I'm running into the this issue on my project and it's causing significant delays. Does this branch solve the issue but is not tested enough for merging to main or is a solution still to be found?

Please see my comments above, the transformers PR https://github.com/huggingface/transformers/pull/28746 should already be fixing this.