noamaanMulla-03/transformers-issues huggingface dataset:
fix: Replaced deprecated `mktemp()` function
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
`tempfile.mktemp()` is deprecated. It's usage can be replaced easily with [NamedTemporaryFile()](https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile),
Reference: https://docs.python.org/3/library/tempfile.html#tempfile.mktemp

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts 

Comments:
Label Leakage in Gemma 2 Finetuning 
Author: hiyouga
Pull request: no

### System Info

- `transformers` version: 4.42.1
- Platform: Linux-5.4.0-147-generic-x86_64-with-glibc2.31
- Python version: 3.10.13
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.30.1
- PyTorch version (GPU?): 2.3.0+cu121 (True)


### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

#### Loss curves

We initially found that the training loss quickly dropped to 0 when fine-tuning the gemma2 models.

![image](https://github.com/huggingface/transformers/assets/16256802/05eb6053-f6e0-485a-bf45-622433e24975)

This phenomenon is very similar to label leakage.

#### Breakpoints

We changed the following lines of `Gemma2DecoderLayer` to the below form.

https://github.com/huggingface/transformers/blob/1c68f2cafb4ca54562f74b66d1085b68dd6682f5/src/transformers/models/gemma2/modeling_gemma2.py#L618-L633

```python
        if self.is_sliding and attention_mask is not None:  # efficient SDPA and no padding
            print("before", attention_mask)
            attention_mask = attention_mask * torch.tril(
                torch.ones_like(attention_mask), diagonal=-self.sliding_window
            )
            print("after", attention_mask)
            if attention_mask.shape[1] <= 1:  # when decoding
                attention_mask = attention_mask[:, -self.sliding_window :]
```

Then we execute the following code to trigger it.

```python
import torch
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-2-9b-it", torch_dtype="auto", device_map="auto", attn_implementation="eager"
)
inputs = torch.tensor([[0, 1, 2]]).to(model.device)
labels = inputs.clone()
outputs = model(input_ids=inputs, labels=labels)
```

The result is:

```
before tensor([[[[ 0.0000e+00, -3.3895e+38, -3.3895e+38],
          [ 0.0000e+00,  0.0000e+00, -3.3895e+38],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00]]]], device='cuda:0',
       dtype=torch.bfloat16)
after tensor([[[[0., -0., -0.],
          [0., 0., -0.],
          [0., 0., 0.]]]], device='cuda:0', dtype=torch.bfloat16)
before tensor([[[[ 0.0000e+00, -3.3895e+38, -3.3895e+38],
          [ 0.0000e+00,  0.0000e+00, -3.3895e+38],
          [ 0.0000e+00,  0.0000e+00,  0.0000e+00]]]], device='cuda:0',
       dtype=torch.bfloat16)
after tensor([[[[0., -0., -0.],
          [0., 0., -0.],
          [0., 0., 0.]]]], device='cuda:0', dtype=torch.bfloat16)
...
```


### Expected behavior

The attention mask should not be all zeros; otherwise, it will use full attention instead of causal attention when calculating the loss, leading to label leakage.

Comments:
Great catch, and sorry that it affected you! 

any update? did  huggingface fix it ?

Yes fixed and released 2 weeks ago!
Mention model_info.id instead of model_info.modelId
Author: Wauplin
Pull request: yes

Related to github.com/huggingface-internal/moon-landing/pull/10713 (server-side, internal link). Let's use `model_info.id` instead of `model.modelId` which is more future-proof cc @julien-c 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32106). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> can we assume the x object here will have both modelID and id

Yes, the `x` type stays the same - in this case `ModelInfo` which is returned by `list_models` and `model_info`.
The only change here is that before the server was returning both `.id` and `.modelId` but not only `.id` will remain. The changes in the PR are to avoid using `.modelId` which is about to disappear. Client-side all objects stay the same except the missing attribute. This is a breaking change but we don't expect it to be problematic.
fix: Fixed raising `TypeError` instead of `ValueError` for invalid type
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
At few places in the code-base we are checking the type of something but raising `ValueError` instead of `TypeError`. I used `ruff` to find these.


Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts 

Comments:
@Sai-Suraj-27 Could you rebase on main? This should fix the hub tests 

> @Sai-Suraj-27 Could you rebase on main? This should fix the hub tests

Done 👍🏻
"use_safetensors" not enforced with "local_files_only", loads bin file
Author: troy-baker-aumni
Pull request: no
Labels: name: Core: Modeling

### System Info

- `transformers` version: 4.41.2
- Platform: macOS-14.5-arm64-arm-64bit
- Python version: 3.11.7
- Huggingface_hub version: 0.23.1
- Safetensors version: 0.4.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.3.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. clone model repo (flan-t5-large)
2. remove `model.safetensors` from target directory
3. run code

```
from transformers import AutoModelForSeq2SeqLM

model = AutoModelForSeq2SeqLM.from_pretrained(
    pretrained_model_name_or_path="path/to/flan-t5-large",
    local_files_only=True,
    use_safetensors=True
)
```

4. observe `from_pretrained` falls back to the `pytorch_model.bin` file and loads the model

### Expected behavior

`from_pretrained` function should raise an error since no safetensors file is present in the target. If this is the target behavior, perhaps this is a feature request rather than a bug, but I find this to be misleading at best or unsafe at worst. Additionally, it is unclear how to manually direct the `from_pretrained` function to only load models with a specific name and extension, or to do so with a file loaded using the safetensors library. It is unclear to the user how a model file is chosen from the dir, so at the very minimum, a user should be able to declare the safetensors file themselves as the model file.

Comments:
Adding to this, this behavior is in direct contradiction to what's written in [the documentation](https://github.com/huggingface/transformers/blob/99c0e5533553d83e02bb327a487e331d5afd9739/SECURITY.md?plain=1#L17)
Add llama3-llava-next-8b to convert_llava_next_weights_to_hf.py
Author: jamt9000
Pull request: no
Labels: name: Feature request

### Feature request

The [convert_llava_next_weights_to_hf.py](https://github.com/huggingface/transformers/blob/main/src/transformers/models/llava_next/convert_llava_next_weights_to_hf.py) script does not support converting the LLaVA-NeXT model based on Llama3-8B, [llama3-llava-next-8b](https://huggingface.co/lmms-lab/llama3-llava-next-8b) (announced [here](https://llava-vl.github.io/blog/2024-05-10-llava-next-stronger-llms/)) meaning it is hard to load the weights of this model with `LlavaNextForConditionalGeneration` from the transformer library.

### Motivation

Adding support would allow loading `llama3-llava-next-8b`, which is a strong multimodal model, with `LlavaNextForConditionalGeneration` included in the transformers lib, allowing full support for transformers functionality (in particular I'd like to use it with vLLM which only implements support for `LlavaNextForConditionalGeneration` rather than the model implementation from the LLaVA repo).

### Your contribution

I have confirmed that modifying the script to add `lmms-lab/llama3-llava-next-8b` and setting
```
        text_model_id = "meta-llama/Meta-Llama-3-8B-Instruct"
        image_token_index = 128256
```

 works and the model output seems sensible (although I'm unsure of the exact subtleties of the conversion and extra tokens). I have made a PR: https://github.com/huggingface/transformers/pull/31395

Comments:
LLaVaNeXT: pad on right if training
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Fixes #32112. The issue didn't mention any bug, as it's using batch-size=1 for training but it led me to this PR. This way we are more confident on which side to pad the inputs. I verified the padding is "right" when training with Trainer and in inference it runs to the block where we infer padding from attn mask

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32134). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Added a test @amyeroberts
LlavaNextVideo always assumes left padding when batch size is 1
Author: zjysteven
Pull request: no
Labels: name: bug

### System Info

Unrelated to this issue

### Who can help?

@zucchini-nlp @Narsil 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

At the moment I don't have a concise script for reproducing, but I think the problem should be clear given the below description. 

Basically I'm trying to finetune the new LlavaNextVideo model. The current implementation (see code below) will always assume `left padding` when the batch size is 1. This will cause a "index out of bound" error when training with batch size being 1, due to the unexpected left padding (instead of right padding) for training.

https://github.com/huggingface/transformers/blob/0fdea8607d7e01eb0e38a1ebeb7feee30a22f0cf/src/transformers/models/llava_next_video/modeling_llava_next_video.py#L565-L576

I can try to put up an example with concrete shape/dims to better show the behavior, but just on a high level I believe the desired behavior is that as long as we are in the training mode (determined by the `_left_padding` and `_right_padding`), we should set `left_padding` to False, regardless of whether batch size is 1.

### Expected behavior

https://github.com/huggingface/transformers/blob/0fdea8607d7e01eb0e38a1ebeb7feee30a22f0cf/src/transformers/models/llava_next_video/modeling_llava_next_video.py#L565-L576

As long as we are in the training mode (determined by the `_left_padding` and `_right_padding`), we should set `left_padding` to False, regardless of whether batch size is 1.

Comments:
@zjysteven hey! Yes, there's an edge case in LLavaNext models when we can't know how the data should be padded from the inputs, this happens when batch size is more than 1 and all inputs withing batch are same length. Usually padding side can be set by `model.padding_side = "right"`

Not sure how this affected training when bs=1, because in that case there's no padding added.

But I think we can also automatically change padding side in model if in train mode for cases when bs > 1. I'll check out it works as expected and open a PR
Remove `trust_remote_code` when loading Libri Dummy
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

The Hub commit [5be9148](https://huggingface.co/datasets/hf-internal-testing/librispeech_asr_dummy/commit/5be91486e11a2d616f4ec5db8d3fd248585ac07a) converted LibriSpeech ASR Dummy from an arbitrary loading script to parquet format. Following this, we can remove the argument `trust_remote_code` when loading the dataset.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31748). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[modelling] remove un-necessary transpose for fa2 attention
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/pull/31166#discussion_r1662573951. Previously, we were performing three shape operations on the `q_proj` tensor:
1. Reshape to `(bsz, tgt_len, num_heads, head_dim)`
2. Transpose to `(bsz, num_heads, tgt_len, head_dim)`
3. Transpose to `(bsz, tgt_len, num_heads, head_dim)`

Clearly 2 and 3 are not required! This PR fixes the modelling code accordingly.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31749). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

The pattern appears in most LLaMA-style attention modules, but here the back-to-back transpose operations **are required**, since there's a step in between the transpose ops:
1. Reshape to `(bsz, tgt_len, num_heads, head_dim)`
2. Transpose to `(bsz, num_heads, tgt_len, head_dim)`
3. Apply rotary pos embeddings (note that this requires the shape produced from 2)
4. Transpose to `(bsz, tgt_len, num_heads, head_dim)`

This leaves only two additional models where there are redundant transpose ops: Idefics2 and Jamba, changes for which I've pushed to the PR.
Add AraBert Model
Author: jadechoghari
Pull request: yes

Goal: Make Ai more open and accessible to all 🤗
Adding AraBert Model - the go-to model for any Arabic NLP task;
(The model is a bit similar to bert_japanese)
https://github.com/aub-mind/arabert

TODO: 
- add testing
- fix jar files problem for Farasa segmenter

Problem facing that a HF could help:
- **Jar File Requirement**: 
  The preprocessing step for some models in AraBert requires the Farasa segmenter, which depends on a jar file that needs to be manually downloaded and included. I'm not sure how fine are HF with adding a java binary file or if I could simply import the library...
`from farasa.segmenter import FarasaSegmenter`
https://github.com/MagedSaeed/farasapy/blob/master/farasa/__base.py#L233

Thanks for the support
Will tag @NielsRogge



Comments:
Hi,

I would recommend to follow this guide to make AraBert usable through the Transformers library: https://huggingface.co/docs/transformers/custom_models, as this is the current preferred way

Will have a look, thanks!

AraBert uses the same architecture as the original BERT implementation. Should I also create a configuration file for it? I noticed that there aren't any separate configuration files for BertJapanese and BertTweet, which also follow the exact BERT architecture

Yes you can create a configuration for it, by copying the entire config
Cohere: Use `diff` tool instead of Copied from mechanism
Author: younesbelkada
Pull request: yes

# What does this PR do?

As per title

cc @ArthurZucker 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31211). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add method to retrieve used chat template
Author: KonradSzafer
Pull request: yes

This PR introduces a new method that allows users to retrieve the chat template. It encapsulates the logic from the existing `apply_chat_template` method, enabling users to access the template without applying it.

Motivation:
- Tracking of used chat templates in evaluation libraries for reproducibility
- Allowing casual users to explicitly know which template is being used

I saw that you plan to introduce simplifications for managing chat templates in the next release (https://github.com/huggingface/transformers/pull/31733). Please let us know if you think this will be relevant, thanks!

@Rocketknight1 @ArthurZucker @clefourrier 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32032). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@KonradSzafer if you pull the latest version of `main` and rebase your PR the failing test should pass and we can merge this!

@ArthurZucker @Rocketknight1 thanks for the review! Done!
Add new quant method
Author: SunMarc
Pull request: yes

# What does this PR do ?

This PR adds a new quantization method.

This PR needs to following to test it :
- this PR from accelerate https://github.com/huggingface/accelerate/pull/2945
- torch nightly and fbgemm-gpu nighlty : https://pytorch.org/FBGEMM/fbgemm_gpu-development/InstallationInstructions.html#fbgemm-gpu-install-libraries
- H100 

### Example:

```python 
from transformers import FbgemmFp8Config, AutoModelForCausalLM, AutoTokenizer

model_name = "meta-llama/Meta-Llama-3-8B"

quantization_config = FbgemmFp8Config()
quantized_model = AutoModelForCausalLM.from_pretrained(
    model_name, device_map="cuda", quantization_config=quantization_config
)

tokenizer = AutoTokenizer.from_pretrained(model_name)
input_text = "What are we having for dinner?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

output = quantized_model.generate(**input_ids, max_new_tokens=10)

print(tokenizer.decode(output[0], skip_special_tokens=True))
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32047). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Merging this ! The failing test is not due to this PR. It also happens for some pr, not sure why.

Error:
```
Run if [ "push" == "push" ]; then
/home/runner/work/_temp/166fca6c-4eaa-4b08-91ec-1648a6ae992f.sh: line 2: unexpected EOF while looking for matching `)'
```

Small request - could you make sure to update the PR title to be more descriptive before merging? This will help both if we need to ever track things down and for preparing the release notes :)
Table question answering pipeline failing to save
Author: daniellok-db
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.43.0.dev0 (installed from source)
- Platform: macOS-14.4.1-arm64-arm-64bit
- Python version: 3.9.13
- Huggingface_hub version: 0.23.5
- Safetensors version: 0.4.2
- Accelerate version: 0.25.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.1 (False)
- Tensorflow version (GPU?): 2.16.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

@muellerzr @amyeroberts 



### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the following script (just saving and loading a pipeline)

```
import transformers

pipe = transformers.pipeline(
    task="table-question-answering", model="google/tapas-tiny-finetuned-wtq"
)

pipe.save_pretrained("test")
```

this gives the exception:
```
ValueError: You are trying to save a non contiguous tensor: `tapas.encoder.layer.0.attention.self.query.weight` which is not allowed. It either means you are trying to save tensors which are reference of each other in which case it's recommended to save only the full tensors, and reslice at load time, or simply call `.contiguous()` on your tensor to pack it before saving.
```

I'm not sure exactly what's going on, but when reverting [this commit](https://github.com/huggingface/transformers/commit/6f40a213eb10e38a5f242d0645519d413d32d798), it saves successfully.

Is there any way to disable superfast init as a workaround?

### Expected behavior

Saving a pretrained pipeline should not fail with an exception.

Comments:
I got around the issue eventually by loading the model separately from the pipeline and specifying `low_cpu_mem_usage=True`, but I'm still curious what the problem is

@daniellok-db Could you try adding `_supports_param_buffer_assignment = False` to the [pretrained model class](https://github.com/huggingface/transformers/blob/d1ec36b94f5ba45fb2423e74074cfedab48cfe73/src/transformers/models/tapas/modeling_tapas.py#L726)? If this works, would you like to open a PR to add this? This way you get the github contribution 

cc @muellerzr Could you look into why this wasn't caught for this model? We'll want to make sure all the models are load/save compatible before the next release 

@amyeroberts thanks! made the PR and manually verified that saving works.
Hub test: avoid repo creation deadlocks
Author: gante
Pull request: yes

# What does this PR do?

see title :)

- Fixed test: `tests/models/auto/test_processor_auto.py::ProcessorPushToHubTester::test_push_to_hub_dynamic_processor`
  - before:
![Screenshot 2024-07-23 at 09 26 38](https://github.com/user-attachments/assets/1fb8db21-2907-4be9-a659-9261aa83676c)
  - this PR:
![Screenshot 2024-07-23 at 09 26 59](https://github.com/user-attachments/assets/11b815ed-75fa-4049-affc-45d849d9d6fa)


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32151). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts Rebasing fixed the persistent error 🤔 Could be related to some older bug, the root PR was a few weeks old. 

Closing the PR

It's due to this commit being included after rebase: https://github.com/huggingface/transformers/pull/32140

@amyeroberts makes complete sense! Thank you :)