noamaanMulla-03/transformers-issues huggingface dataset:
LLaVA-NeXT-Video: fix generation with cache
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes generation for llava-next-video. Apparently started failing after we moved to cache class but some parts of the code were not modified. Checked all llava models, others are working since check is done on a different condition.

Yes, we can start using `cache_position` and rely on that, but we should note that `cache_position` for VLMs will not be correct and will contain positions only for text tokens. Adding support for cache position will come in the next PR, which is in progress. We;ll have to deprecate many things before we can get rid of the current checks to "merge or expand" 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32527). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Actually we should have VLM generation tests soon (not very soon), but for now maybe I'll try and make a very dummy test for all VLMs. It is annoying that fast tests for VLMs don't catch most basic bugs

Added one test for generation, I will see if we can start integrating test for VLMs before the refactoring is done. The last time it forces us to add many if/elses so we gave up until it is all standardized

Nah, both are equally valid since current VLMs doesn't support speculative decoding. Let's use the PR from yesterday which has this and another fix, I'll close this one. The current state of main already has it fixed due to the recent refactor
VLMs: small clean-up for cache class
Author: zucchini-nlp
Pull request: yes
Labels: name: run-slow

# What does this PR do?

VideoLLaVa was generating garbage in beam search as reported by some users after moving to the new cache format. Thsi PR fixes it as we no longer will have `past_key_values is None` in any generation step. Except for when caching isn't enabled at all.

Also a little clean up on VLMs, after removing old cache format for all major LLMs we no longer need the `reorder cache` method

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32417). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yep, after resolving conflicts the fix wasn't needed anymore. I'll rename PR and merge it :)
pip install error: "regex_3/_regex.c:48:10: fatal error: Python.h: No such file or directory"
Author: seyuboglu
Pull request: no

When pip installing pytorch-pretrained-bert on ubuntu and getting the following error:

```
    regex_3/_regex.c:48:10: fatal error: Python.h: No such file or directory
     #include "Python.h"
              ^~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/.../env/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pogvvuk9/regex/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pogvvuk9/regex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-19j41i0r/install-record.txt --single-version-externally-managed --compile --install-headers /home/.../env/include/site/python3.7/regex Check the logs for full command output.
```

My python version is ```Python 3.7.4 (default, Jul  9 2019, 15:11:16)
[GCC 7.4.0] on linux```.

I'm able to pip install other packages without problems. 
Has anyone run into a similar issue?

Comments:
Hi @seyuboglu I think you need to install the Python Dev package on your distribution. For Ubuntu >= 18.04 this should be possible with `apt install python3.7-dev` :)

Thank you @stefan-it! That did the trick. Any idea why this is necessary to install pytorch-transformers in particular? 

@stefan-it do you know how to make this happen on on amazon linux2, I have tried a bunch of things am am getting same error. I think i already installed  a python3-development package with yum. but when i tried python3.7-dev it said no package located. 

@antleypk Could you try to use `yum install python3-devel` instead?

Thanks @stefan-it 

I solved it last week and should have updated thread. 

To anyone else that may come here; this solution is for "amazon linux2"

my current setup.sh scripts looks like this:

     sudo yum install python3.x86_64 -y
     sudo yum install python3-devel.x86_64 -y

Since getting adding the bottom line I have been able to install every package that I tried to install.

I built the answer based on this solution: 
https://stackoverflow.com/questions/43047284/how-to-install-python3-devel-on-red-hat-7

and found this solution based on this original post. 




@antleypk Thanks a lot! That's really helpful for me. My env is CentOS8.0 and python3.11.7.
add back the position ids
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #32549, the position ids where not taken into account before, had to fix the `prepare_inputs_for_generation`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32554). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Google RecurrentGemma Models don't work in Transformers 4.43 anymore
Author: dipanjanS
Pull request: no
Labels: name: bug

### System Info

Transformers version 4.43.3 causes this error, works fine in 4.42

```
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = "google/recurrentgemma-2b-it"
dtype = torch.bfloat16
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="cuda",
    torch_dtype=dtype,
)

chat = [
    { "role": "user", "content": "Explain what is AI in 3 bullet points" },
]
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)

inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device),
                         max_new_tokens=150)
print(tokenizer.decode(outputs[0]))
```

Gives the following error now: 

`RecurrentGemmaForCausalLM.forward() got an unexpected keyword argument 'position_ids'`

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Already mentioned above.

### Expected behavior

The prompt should run without errors just like in version 4.42

Comments:
Opening a PR for a fix! 

It was broken by #31549 😅 

Thanks for the prompt fix, much appreciated!
Generate: unify `LogitsWarper` and `LogitsProcessor`
Author: gante
Pull request: yes

# What does this PR do?

`LogitsProcessor` and `LogitsWarper` were originally created as two separate classes, with `LogitsWarper` being a `LogitsProcessor` that should only be applied to sample-based decoding methods. In practice, they are exactly the same.

Having two separate classes meant that there was undesired code duplication: two functions to instantiate these objects, two separate pieces of logic to apply these processors, etc.

⚠️ Perhaps more importantly, given that the order of the logit processors is important, having two separate types of objects results in needing custom logic if we want to interleave the two types of instances. Since we don't want custom logic, this PR is a requirement for an ongoing project integration.

This PR:
- removes all internal uses of `LogitsWarper` and related logic, moving it to the logic related to `LogitsProcessor`
- keeps `LogitsWarper` public with a deprecation warning, for BC

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32626). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[tests] make test_sdpa_equivalence device-agnostic
Author: faaany
Pull request: yes
Labels: name: run-slow

## What does this PR do?
This unit test should not only work on GPU, e.g. below is an evidence on XPU:
```bash
==================================================== PASSES =====================================================
_____________________________________ GemmaModelTest.test_sdpa_equivalence ______________________________________
--------------------------------------------- Captured stderr call ----------------------------------------------
`config.hidden_act` is ignored, you should use `config.hidden_activation` instead.
Gemma's activation function will be set to `gelu_pytorch_tanh`. Please, use
`config.hidden_activation` if you want to override this behaviour.
See https://github.com/huggingface/transformers/pull/29402 for more details.
============================================ short test summary info ============================================
PASSED tests/models/gemma/test_modeling_gemma.py::GemmaModelTest::test_sdpa_equivalence
```

Comments:
> LGTM - thanks!
> 
> Two things for us to be able to merge:
> 
> * Move the PR out of drafts
> * A run on slow tests. Could you push a commit with the message `[run_all]`?

Hi amy, I am not sure what I should do with the second point. Do you mean that I should add a RUN-ALL Message in our chat here?

@faaany Sorry, I wasn't clear. I meant you should push a commit with this message i.e. `git commit --allow-empty -m [run_all]`

> @faaany Sorry, I wasn't clear. I meant you should push a commit with this message i.e. `git commit --allow-empty -m [run_all]`

thx! done with empty commit. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32520). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for running with the commit! As Yih-Dar pointed out in the other PR, the tests are essentially skipped. However, since  this isn't changing any logic I think it's fine, so merging. Thanks again for working on this!
Cache: use `batch_size` instead of `max_batch_size`
Author: gante
Pull request: yes

# What does this PR do?

Renames the input argument `max_batch_size`, present in static-shaped caches, to `batch_size`. `max_batch_size` is imprecise: the cache needs the EXACT batch size being used. 

The imprecise variable name and description was a source of issues, e.g. [here](https://github.com/huggingface/transformers/pull/31772#issuecomment-2286151919).

NOTE: while it is technically feasible to accept smaller batch sizes in static-shaped caches, we would be slicing the cache at each layer. Slicing is an expensive operation, and the point of using static-shaped caches is to be fast. In other words, we would be enabling a silent incorrect usage of the class 🤗

______________________________________

✅ all chances are backwards compatible, and the user will only see a warning if passing the batch size through the deprecated keyword argument
```py
from transformers import AutoConfig, StaticCache

config = AutoConfig.from_pretrained("gpt2")

# No warnings
StaticCache(config, 8, 100, "cpu")
StaticCache(config=config, batch_size=8, max_cache_len=100, device="cpu")

# Warnings
StaticCache(config=config, max_batch_size=8, max_cache_len=100, device="cpu")
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32657). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
improve _get_is_as_tensor_fns
Author: zrr1999
Pull request: yes

# What does this PR do?

When using VivitImageProcessor, there will be a warning for _get_is_as_tensor_fns,

```
/home/zrr/workspace/emotion-recognition-project/.venv/lib/python3.11/site-packages/transformers/feature_extraction_utils.py
:142: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list 
to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at 
../torch/csrc/utils/tensor_new.cpp:278.)
```
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?


<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

@ArthurZucker
@amyeroberts
@sanchit-gandhi

 -->


Comments:
@zrr1999 Could you run `make fixup` to resolve the quality checks? 

Okay, I just fixed it.
Fix quantization w/ DeepSpeed not working
Author: muellerzr
Pull request: yes

# What does this PR do?

This PR tweaks the logic check introduced in https://github.com/huggingface/transformers/pull/32299 to specifically exclude when Zero-3 is enabled and we're using a quantization method (which skips this chunk of the zero3 init use)

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc can you check if what I've done here makes sense given the comment we're trying to fix?

https://github.com/huggingface/transformers/pull/32299#issuecomment-2284975360

I believe so and I think the test makes sense, but you're quant eyes would be appreciated :) 



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32640). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This fixed the issue with finetuning I reported [here](https://github.com/huggingface/transformers/pull/32299#issuecomment-2284975360), thanks!

For my own understanding - why doesn't the user need to create the model after TrainingArguments in the quantized case but does in the unquantized case? 

Quantized weights aren't thrown into the `zero_init` context manager and instead are using `init_empty_weights` (I think this answers both your questions)

Closing since we're doing a full revert
Uniformize processor kwargs of siglip
Author: leloykun
Pull request: yes

# What does this PR do?

- Uniformizes kwargs for SigLip processors as discussed in https://github.com/huggingface/transformers/issues/31911
- Already has backwards compatibility support

TODO:
- [ ] add tests

Fixes # (issue)

- https://github.com/huggingface/transformers/issues/31911

## Who can review?

@zucchini-nlp @molbap @NielsRogge 

Comments:
closing as duplicate

The contents of this PR has now been merged to https://github.com/huggingface/transformers/pull/32845
Revert PR 32299, flag users when Zero-3 was missed
Author: muellerzr
Pull request: yes

# What does this PR do?

Reverts https://github.com/huggingface/transformers/pull/32299 until I can come up with a better solution. 

TL;DR:

1. This broke quantization, which was attempted to fix in this PR: https://github.com/huggingface/transformers/pull/32640
2. However, this breaks Zero-3 inference plainly, so for now disabled while leaving context in the test. 

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32851). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

None that I can *tell*, but our DS CI needs work to actually make the error logs readable. I'll see what I can do.
Reduce the error log when using core models that need their weights renamed, and provide a step forward
Author: muellerzr
Pull request: yes

# What does this PR do?

https://github.com/huggingface/transformers/pull/31654 introduced a (good) warning for users when we internally rename certain parameters in a model (`gamma` and `beta`). 

The flag however was extremely verbose and doesn't include a path forward for users who have no control over the model (like `bert-base-cased`).

This PR:
1. Introduces a new `info_once` to the `logger` for when we want something to be at the `INFO` level (since general users don't need this warning)
2. Reduces this warning down to `info`
3. Changes the warning to be more transparent to the user towards a solution

Old warning:
```
A parameter name that contains `beta` will be renamed internally to `bias`. Please use a different name to suppress this warning.
A parameter name that contains `gamma` will be renamed internally to `weight`. Please use a different name to suppress this warning.
A parameter name that contains `beta` will be renamed internally to `bias`. Please use a different name to suppress this warning.
A parameter name that contains `gamma` will be renamed internally to `weight`. Please use a different name to suppress this warning.
...
```

New warning:
```
This model (`bert-base-cased`) contains parameters that have been renamed internally (a few are listed below but more are present in the model):
* `bert.embeddings.LayerNorm.beta` -> `bert.embeddings.LayerNorm.bias`
* `bert.embeddings.LayerNorm.gamma` -> `bert.embeddings.LayerNorm.weight`
If you are using a model from the Hub, consider submitting a PR to adjust these weights and help future users.
```
Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts


Comments:
Maybe it should be "adjust weight names"? 🤔 Open to ideas or leaving it as it is

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32656). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Make beam_constraints.Constraint.advance() docstring more accurate
Author: alex-calderwood
Pull request: yes

# What does this PR do?
The abstract base class Constraint.advance() docstring indicated that it could only return an integer token_id, when in fact it can return a list of token_ids.


Fixes #32668
https://github.com/huggingface/transformers/issues/32668


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante @stevhliu 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32674). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.