noamaanMulla-03/transformers-issues huggingface dataset:
PixtralProcessor always returns outputs of length 1
Author: Infernaught
Pull request: no
Labels: name: Good First Issue

name: bug

name: Vision

name: Multimodal

### System Info

- `transformers` version: 4.45.2
- Platform: Linux-5.4.0-1113-oracle-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.23.2
- Safetensors version: 0.4.5
- Accelerate version: 0.34.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.2.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: False
- Using GPU in script?: True
- GPU type: NVIDIA RTX A5000

### Who can help?

@ArthurZucker @amyeroberts Hello! I'm working on finetuning a VLM, and during my dataset preprocessing, I'm noticing that the PixtralProcessor always returns the input ids for only the first example in a batch. I think this is being caused by the [lines here](https://github.com/huggingface/transformers/blob/3f06f95ebe617b192251ef756518690f5bc7ff76/src/transformers/models/pixtral/processing_pixtral.py#L206), which are aggregating all of the images into a length 1 list of lists, which is messing with the iteration over the zip [here](https://github.com/huggingface/transformers/blob/3f06f95ebe617b192251ef756518690f5bc7ff76/src/transformers/models/pixtral/processing_pixtral.py#L236). Is this unintended behavior or am I doing something wrong?

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
processor = AutoProcessor.from_pretrained("mistral-community/pixtral-12b")

# Define prompts as a list of 50 text prompts and images as a list of 50 decoded images
batch = processor(prompts, images, padding=True, return_tensors="pt") # Returns a batch containing only the first of the 50 elements
```

### Expected behavior

I would expect it to return the outputs corresponding to all 50 prompts and images.

Comments:
Ah this is not expected, would you like to open a PR for a fix? 🤗 

Sure let me give it a shot
fix pixtral processor
Author: molbap
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Should fix https://github.com/huggingface/transformers/issues/34204 . 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34486). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Tests (nonslow, just launched the slow ones) are green - there was a mismatch in the tests between images per batch and images per sample which I think I fiixed, added a test as well for previous issue. cc @ArthurZucker for review

all green incl. slow now - fixed slow tests that were broken before (well one fix for a wrong config key at init, one that actually can't work, and a skip for torchscript) lmk!
Call `.destroy()` on `DeepSpeedEngine` somewhere post training
Author: chiragjn
Pull request: no
Labels: name: trainer

name: Feature request

name: DeepSpeed

### System Info

transformers==4.36.2
accelerate==0.25.0
deepspeed==0.12.5

### Who can help?

I was using deepspeed stage 2 with Trainer and accelerate and at the end of training when the Trainer has been garbage collected, I noticed my GPU VRAM was not clearing even after aggressively calling `gc.collect()` and `torch.cuda.empty_cache()`

I spent some time debugging and narrowed it down to deepspeed optimizer not removing hooks on pytorch tensors.
I have submitted a PR on Deepspeed: https://github.com/microsoft/DeepSpeed/pull/4858 
But to invoke this logic `engine.destroy()` must be called in some place post-training

For now, I am manually calling it outside the trainer post-training and can confirm it works, would be nice if Trainer can take care of it or there is some note in the docs.

@pacman100

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

- Train any model with Zero 2 + gradient accumulation, delete and let the trainer garbage collect, model parameters would still linger around in the GPU memory


### Expected behavior

GPU memory should be reclaimable post training

Comments:
Gentle ping @pacman100 for your thoughts on this feature addition

This was resolved in https://github.com/huggingface/accelerate/pull/2716
Use torch 2.5 in scheduled CI
Author: ydshieh
Pull request: yes

# What does this PR do?

It's time to switch.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34465). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

OK
Couldn't find transformer version 5.0.0 that supports fall back support for manual attention implementation.
Author: kzos
Pull request: no

When I tried to add output_attentions=True, it says the work around in 'eager' can expose attention output, but its available in v5.0.0


`LlamaModel is using LlamaSdpaAttention, but `torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to the manual attention implementation, but specifying the manual implementation will be required from Transformers version v5.0.0 onwards. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.
`Starting from v4.46, the `logits` model output will have the same type as the model (except at train time, where it will always be FP32)

How do i get v5.0.0 (when I check repo, I could find: release tags v4.5.1, dev note v4.6.x)?

Comments:
Hey @kzos! This is a FutureWarning and we don't have v5.0 released yet. It says that after releasing v5.0 we will no longer fallback to "eager" attention and you will have to `AutoModel.from_pretrained(model_id, attn_implementation="eager")` if you want to get attention scores in the output. Otherwise there is no need to indicate "eager" attention usually
Include passthrough to free_memory
Author: muellerzr
Pull request: yes

# What does this PR do?

Per request in https://github.com/huggingface/accelerate/pull/2716, this PR introduces a passthrough `Trainer.free_memory` function. 

Now, since most of the refs are built inside the `Trainer` the user really only needs to include/pass through a `model` rather than needing *everything else*

Secondary fix to https://github.com/huggingface/transformers/issues/28178


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @pacman100 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30549). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Seems like this got auto closed, this should have been merged, right?
@amyeroberts @pacman100 

Thanks! I've opened https://github.com/huggingface/transformers/pull/34519 instead
UPDATE Documentation for #TRANSLATING.md Documentation into Multiple Languages.(Changes made)
Author: anshumangahlot
Pull request: yes
Labels: name: Documentation

This PR introduces a guide for translating the Transformers documentation into multiple languages as part of the project’s effort to make machine learning more accessible globally.

### Key Changes:

- **New Translation Instructions**: Added detailed step-by-step instructions for contributing translations, covering everything from opening issues to forking the repository, and translating relevant files.

	

- **File Organization**: Included a guide on how to copy the English documentation to a new language folder using ISO 639-1 or ISO 639-2 language codes.

	

- **_toctree.yml Guidance**: Provided instructions on editing or creating the _toctree.yml file to correctly render the documentation’s table of contents for the new language.

	

- **Collaboration Encouragement**: Emphasized the importance of collaboration, inviting contributors to share glossaries and tag team members for assistance.

### Files Modified:

- Added docs/source/translation_guide.md: This file contains the full translation guide, with detailed instructions on getting started, collaborating, and submitting translations.

**Additional Notes:**

- The guide encourages new contributors to open issues before starting a translation, ensuring no duplicate efforts.

- Resources and tips for effective translation are included to help users maintain consistent terminology.

- I’ve added links to relevant documentation, including language code references and GitHub commands.


Comments:
cc @stevhliu

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34226). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add DistilBERTGeneration comparable to BertGeneration
Author: jsilter
Pull request: no
Labels: name: Good Second Issue

# 🚀 Feature request

I noticed the new `BertGeneration` class, which uses BERT-style models as both encoder and decoder, as well as the more general `EncoderDecoder` class. This is all great stuff! It would also be great to be able to use distilled models. I believe this is possible for the encoder, but for the decoder a language head must be added.

Since DistilBert is implemented as its own model, and not as a BertModel, I don't think it's possible (or at least it's not easy) for the end user to do this. At least not loading pretrained models, since any pretrained model needs to be a type approved by `AutoModelForCausalLM`.

## Motivation

Same motivation as using distilled models in general. Same results at higher speed, this time applied to an `EncoderDecoder` model.

## Your contribution

Happy to be an alpha tester for this feature


Comments:
Hey @jsilter - yes we could definitely add a `DistilForCausalLM` model. I think instead of doing something similar to `BertGeneration` it would be easier to just add a `DistilBertForCausalLM` to `modeling_distilbert.py` similar to `BertLMHeadModel` or `RobertaForCausalLM`. This could actually be an interesting `Good Second Issue`. If someone is interested in opening a PR - I'd be more than happy to provide some guidance :-)

Hi @patrickvonplaten, I would love to work on this if it is still possible?

Hey @KMFODA - yes absolutely :-) Do you want to open a PR? I think we can very analogues to `BertLMHeadModel` add a `DistilBertForCausalLM` model in `modeling_distilbert.py`.

Great! Will open up a PR and start adding a `DistilBertForCausalLM` model into `modeling_distilbert.py` and get back to you if I have any issues :)

Hi @patrickvonplaten, I've built the `DistilBertForCausalLM` class into `modelling_distilbert.py` and can run it on the example used in both the `BertLMHeadModel` and the `RobertaForCausalLM` and the outputs look fine. Other than this example, are there any other tests I can run to check it's working as expected?


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Hello @patrickvonplaten! Would this still be a valued contribution? I'm currently a graduate student at CU Boulder studying NLP, Formal Methods and Program Synthesis, and this issue sounds interesting to me.
tokenizer's `apply_chat_template` doesn't preserve trailing newlines
Author: lone17
Pull request: no
Labels: name: bug

name: Chat Template

### System Info

- `transformers` version: 4.46.0
- Platform: Linux-5.15.0-122-generic-x86_64-with-glibc2.35
- Python version: 3.12.7
- Huggingface_hub version: 0.26.1
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.0+cu124 (True)

### Who can help?

@Rocketknight1 @itazap @ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The current behaviour of `apply_chat_template` is removing the last trailing newline in the chat template, even if it has more than one newlines.

```python3
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")

tokenizer.chat_template = """
user
hello


assistant
"""

print(repr(tokenizer.chat_template))
# '
user\nhello
\n
assistant\n'
print(repr(tokenizer.apply_chat_template(None, tokenize=False)))
# '
user\nhello
\n
assistant'

tokenizer.chat_template = """
user
hello


assistant


"""

print(repr(tokenizer.chat_template))
# '
user\nhello
\n
assistant\n\n\n'
print(repr(tokenizer.apply_chat_template(None, tokenize=False)))
# '
user\nhello
\n
assistant\n\n'

```

Not sure if this is the intended behaviour but I didn't find any discussion/document that mentions this. If this was indeed intended, I would like to learn why.

From my experience, there are models that are quite sensitive to this last newline token. Besides, most users would be unaware of this behaviour, which might cause unexpected outcomes.

I have traced the code and found out that this was due to `jinja2` default configs. It can be fixed by simply setting `keep_trailing_newline=True` in the below code. After you guys confirm that this is a bug, I can make a PR if needed.

https://github.com/huggingface/transformers/blob/1d063793318b20654ebb850f48f43e0a247ab7bb/src/transformers/utils/chat_template_utils.py#L422-L424

### Expected behavior

- `apply_chat_template` should preserve that `chat_template`
- If there is any modification to the template then it should be documented.

Comments:
Hi @lone17, this could probably be documented better, but **in general, we recommend putting important whitespace inside `{{ blocks }}`**. If you want to print newlines at the end of your template, add a `{{- "\n\n" }}` block.

Hi @Rocketknight1, so it seems that the behaviour was intended. In that case, let me close this and please update the docs to mention this behaviour. Many thanks.
ValueError: LlavaForConditionalGeneration does not support an attention implementation through torch.nn.functional.scaled_dot_product_attention yet. Please open an issue on GitHub to request support for this architecture: https://github.com/huggingface/transformers/issues/new
Author: 1106280506Hx
Pull request: no
Labels: name: Feature request

name: SDPA

processor = AutoProcessor.from_pretrained("/gemini/data-2/data/llava")
model = AutoModelForPreTraining.from_pretrained("/gemini/data-2/data/llava",load_in_4bit=True,bnb_4bit_compute_dtype=torch.float16,low_cpu_mem_usage=True,attn_implementation="sdpa").to("cuda") 




Comments:
cc @younesbelkada for reference 

can you tell me how to sovle it?

Hi @1106280506Hx, 

As the error suggests, SDPA isn't supported in Llava on the version of transformers being run. However, just looking it up, #28107 was merged in a few days ago. You should be able to resolve this by running the dev version of transformers - installing from source: 
`pip install git+https://github.com/huggingface/transformers`

Your answer is meaningful, thanks for your quick reply  @amyeroberts

LLaVA supports sdpa now for llm backbone and for vision tower, closing as completed
Include other tokenizers/image processors in Llava
Author: muhark
Pull request: no
Labels: name: Good Second Issue

name: Feature request

### Feature request

Generalize the functionality in [processing_llava.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llava/processing_llava.py) to include other tokenizers and image processors.

### Motivation

The current implementation of the LlaVA processor only accepts Llama as the tokenizer. Given the extensibility of the framework, this restriction should not be there.

### Your contribution

I am happy to write the code and make the PR. We have done this for our own internal research code and there is an example file in our `llava-gemma-2b` model ([link](https://huggingface.co/Intel/llava-gemma-2b/blob/main/processing_llavagemma.py)).

Comments:
I am not sure there has been other Llava models that don't use llama tokenizer no? 

@ArthurZucker thank you for the fast response!

There are a few examples I know of:

- TinyLlaVA uses phi-2 (but requires users to install their package)
- We have just uploaded a Gemma-based LLaVA model

Is there a strong reason for restricting the LlaVA Processor to using llama/clip?

There is none! It's just that we base it on usage! Could you link to the gemma based model? 
Feel free to open a PR


FYI @NielsRogge 

Link to gemma-based model: [Intel/llava-gemma-2b](https://huggingface.co/Intel/llava-gemma-2b)

I'm happy to open the PR :)


Feel free to do so and link it to this issue!

Hello, am just wondering how to append -200 image tokens to nay tokenizer and reuse LLavaProcessor?

Hey, you can simply use `tokenizer.add_tokens(["toke1", "tok2", .........])` 

LLaVA now supports any type of tokenizer and LLM backbone, so should be completed. Feel free to reopen if it needs anything else
add diffllama with modular transformers
Author: bzantium
Pull request: no
Labels: name: New model

### Model description

- this is almost first try to use modular transformers
- it is implementation for differential transformers described in this [paper](https://arxiv.org/abs/2410.05258)
- the architecure cancels noise by subtracting attention scores as the difference between two separate softmax attention maps

### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

https://arxiv.org/abs/2410.05258

Comments:
Hi bzantium,

I would be interested in making contributions for this issue. Could I be assigned to this task.

Thank you.

Regards,
Huzaifa Jawad

Hi! you can contribute [here](https://github.com/huggingface/transformers/pull/34083)! Thanks for the interest 👍 
to: @huzaifa-jawad367
Github Actions automatically run and fail when sync fork
Author: Imss27
Pull request: no
Labels: name: Discussion

name: bug

### System Info

N/A

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Sync fork in my repo will cause such problem and I keep getting emails about failed CI jobs. 

### Expected behavior

Github actions CI should not run automatically for my forked repo.

Comments:
I think that's more an issue with settings in your git clone no? 

If you're aware of any settings we could change to make your life simpler, let us know!

That's strange. Actually I just forked the repo and didn't commit anything.🤔

Anyway, just wonder if anyone has similar issues, I am not a CI expert, sorry can't be more helpful.

I attached a picture for your reference, looks like it regards some commits as my commits.
![ci](https://github.com/user-attachments/assets/a00fddd5-3308-49c5-b312-724b9ed40e56)

Feel free to close this issue if we can't find the reason for now.
Add `post_process_depth_estimation` for GLPN
Author: alex-bene
Pull request: yes
Labels: name: Vision

name: Processing

# What does this PR do?

This PR adds a post_process_depth_estimation method for the image processor of GLPN (thus fixing its inference via the depth pipeline) and updates its docs.

Fixes #34300 and [comment](https://github.com/huggingface/transformers/pull/32550#issuecomment-2429461037) on #32550

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).

## Who can review?
@ydshieh @qubvel @amyeroberts 

Comments:
Thanks!

Could you remove the following block, thanks!

(in `tests/models/glpn/test_modeling_glpn.py`)
```python
    @unittest.skip(reason="Failing after #32550")
    def test_pipeline_depth_estimation(self):
        pass

    @unittest.skip(reason="Failing after #32550")
    def test_pipeline_depth_estimation_fp16(self):
        pass
```

They pass now indeed.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34413). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker  ready to merge?