noamaanMulla-03/transformers-issues huggingface dataset:
Hotfix-change-ci
Author: ArthurZucker
Pull request: yes

# What does this PR do?
We ought to test the list of extanded tests: https://app.circleci.com/pipelines/github/huggingface/transformers/92007/workflows/f32a5969-555a-404f-9048-e631abd0067e/jobs/1203774/parallel-runs/1?filterBy=ALL 

failed because it produce 3 parallelism : we were computing the length of the string `test/intagration/....` // 10 -> 3 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30669). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix llava next tie_word_embeddings config 
Author: SunMarc
Pull request: yes

# What does this PR do ?
This PR fixes the llava next config. We set `tie_word_embeddings` to `False` since we don't tie the weights at all (actually the `tie_weights` is redefined to call the `language_model.tie_weights() instead`). 
This fixes the following issue https://github.com/huggingface/transformers/issues/30001 where we get a warning about untied weights. This happens because we don't find any tied weights even though `tie_word_embeddings` is set to `True`. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30640). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Trainer with resume_from_checkpoint does not work with multiple Peft Adapters
Author: claralp
Pull request: no
Labels: name: trainer

name: Quantization

name: PEFT

### System Info

Since it it possible to have multiple Peft adapters in the same model, it should also be possible to resume a training of such models from checkpoint with [transformers.Trainer.train(resume_from_checkpoint=True|"path")](https://huggingface.co/docs/transformers/main_classes/trainer#transformers.Trainer.train)  
This is needed to resume e.g. DPO/KTO trainings loaded with 2 adapters, as in [load-the-adapter-twice](https://huggingface.co/docs/trl/dpo_trainer#using-option-3---load-the-adapter-twice).

This is due to the fact that if multiple adapters exist, their weights get saved in subdirectories of the checkpoint, not directly in there. So no `adapter_model.bin` or `adapter_model.safetensors` can be found.

Anyone working on this or should I come up with a solution?


### Who can help?

@ArthurZucker @younesbelkada @muellerzr 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from https://huggingface.co/docs/trl/dpo_trainer#using-option-3---load-the-adapter-twice
``` python
# Load the base model.
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    llm_int8_threshold=6.0,
    llm_int8_has_fp16_weight=False,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
)
model = AutoModelForCausalLM.from_pretrained(
    "mistralai/mixtral-8x7b-v0.1",
    load_in_4bit=True,
    quantization_config=bnb_config,
    attn_implementation="flash_attention_2",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
model.config.use_cache = False

# Load the adapter.
model = PeftModel.from_pretrained(
    model,
    "/path/to/peft",
    is_trainable=True,
    adapter_name="train",
)
# Load the adapter a second time, with a different name, which will be our reference model.
model.load_adapter("/path/to/peft", adapter_name="reference")

# Initialize the trainer, without a ref_model param.
dpo_trainer = DPOTrainer(
    model,
    ...
    model_adapter_name="train",
    ref_adapter_name="reference",
)
```
After a checkpoint is saved, interrupt the training and resume with:
`dpo_trainer.train(resume_from_checkpoint=True)`  

But this throws the following error: `ValueError(f"Can't find a valid checkpoint at {resume_from_checkpoint}")`   

### Expected behavior

the training should resume

Comments:
also interesting for @lewtun @kashif
model_max_length default parameters are missing in transformers>=4.40.0
Author: helpmefindaname
Pull request: no

### System Info

- `transformers` version: 4.40.0
- Platform: Windows-11-10.0.22631-SP0
- Python version: 3.12.3
- Huggingface_hub version: 0.23.0
- Safetensors version: 0.4.3
- Accelerate version: 0.29.3
- Accelerate config:    not found
- PyTorch version (GPU?): 2.3.0+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker @younesbelkada because of tokenization/text models
@LysandreJik as the bug was introduced by https://github.com/huggingface/transformers/pull/29112

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

install tokenizers>=4.40.0:
`pip install "transformers>=4.40.0"`

run python:
```python
from transformers import AutoTokenizer;

tokenizer = AutoTokenizer.from_pretrained('distilbert-base-uncased')
max_length = tokenizer.model_max_length

print(max_length)
```

prints `1000000000000000019884624838656` which indicates infinity sequence length, however distilbert-base-uncased has a max sequence length of 512
### Expected behavior

The code above should print `512` like it does in `transformers==4.39.3`

### Details

This bug is a direct regression of https://github.com/huggingface/transformers/pull/29112 which refactors the way default configurations are stored. For the `PretrainedConfig` classes, the config-maps were moved and deprecated, but for the Tokenizer classes, the default values were just removed.

I suppose a quick fix would be to also create deprecated default configurations for the Tokenizers (analogous to the deprecated config maps). I can work on that, if you accept this solution


Comments:
Thanks @helpmefindaname, it seems the update to this tokenizer's config file wasn't merged. I just merged it: https://huggingface.co/distilbert/distilbert-base-uncased/discussions/12

It seems a few others from distilbert weren't merged. I just merged them. Thanks again for the heads-up!
Inconsistency with transformer pipeline results
Author: abhijeetk597
Pull request: no

### System Info

- `transformers` version: 4.38.2
- Platform: Linux-5.15.133+-x86_64-with-glibc2.31
- Python version: 3.10.13
- Huggingface_hub version: 0.21.4
- Safetensors version: 0.4.2
- Accelerate version: 0.28.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.2+cpu (False)
- Tensorflow version (GPU?): 2.15.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.8.2 (cpu)
- Jax version: 0.4.25
- JaxLib version: 0.4.25
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import pipeline, AutoTokenizer, AutoModelForSequenceClassification

MODEL = f"cardiffnlp/twitter-roberta-base-sentiment"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL)

sent_pipeline = pipeline("sentiment-analysis",model=model, tokenizer=tokenizer)

import random
random_text = random.choice(df["Text"])

print(random_text)
sent_pipeline("random_text")
```
![Screenshot 2024-03-30 052904](https://github.com/huggingface/transformers/assets/138308825/58f31510-7f55-4d91-8e0a-2e63e323d5e0)

## I tried same thing with different model but got same kind of result.
![Screenshot 2024-03-30 053110](https://github.com/huggingface/transformers/assets/138308825/a317ead7-5b32-4b67-9277-51318b2ee09c)

## [Kaggle Notebook Link](https://www.kaggle.com/code/abhijeetk597/sentiment-analysis-using-nltk-and-transformers)

### Expected behavior

Sentiment scores should vary as per text given.

Comments:
You pass the string "random_text" but not the variable's (`random_text`) content. That's why you get the same score.

Try passing it like `sent_pipeline(f'{random_text}')`. So in total something like:
```python
from transformers import pipeline, AutoTokenizer, AutoModelForSequenceClassification

MODEL = f"cardiffnlp/twitter-roberta-base-sentiment"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForSequenceClassification.from_pretrained(MODEL)

sent_pipeline = pipeline("sentiment-analysis",model=model, tokenizer=tokenizer)

import random
random_text = random.choice(df["Text"])

print(random_text)
# here is the change
sent_pipeline(f"{random_text}")
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Inconsistency between torch_dtype in model.config and torch_dtype of model parameters
Author: efsotr
Pull request: no

### System Info

- `transformers` version: 4.39.2
- Platform: Linux-4.15.0-213-generic-x86_64-with-glibc2.27
- Python version: 3.9.16
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.2
- Accelerate version: 0.23.0
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: MULTI_GPU
        - mixed_precision: no
        - use_cpu: False
        - debug: False
        - num_processes: 8
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: False
        - main_training_function: main
        - downcast_bf16: False
        - tpu_use_cluster: False
        - tpu_use_sudo: False
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker @younesbelkada


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
import torch
from transformers import LlamaForCausalLM, LlamaConfig, AutoModelForCausalLM

config = LlamaConfig(num_hidden_layers=2, hidden_size=16, num_attention_heads=16, torch_dtype="float16")
model = AutoModelForCausalLM.from_config(config, torch_dtype=torch.bfloat16)
print(model.get_input_embeddings().weight.dtype, model.dtype)
print(model.config.torch_dtype)
```

Running preceding code, then we will get 
```
torch.float32 torch.float32
torch.float16
```

### Expected behavior

```
torch.float16 torch.float16
torch.float16
```

Comments:
Hey! I asm using main but getting this: 
<img width="742" alt="image" src="https://github.com/huggingface/transformers/assets/48595927/57a04e98-8c02-48e6-a32f-e661c96105fa">

which is already a bit better and expected. The config value was not overwritten. However if you save it, it will be overwritten 😉 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Minor difference in rouge scores for BART evaluation on summarization task
Author: Madhumitha-MCW
Pull request: no

### System Info

transformers version: 4.39.0
Platform: Linux-5.4.0-167-generic-x86_64-with-glibc2.31
Python version: 3.10.13
Huggingface_hub version: 0.21.4
Safetensors version: 0.4.2
Accelerate version: 0.28.0
Accelerate config: not found
PyTorch version (GPU?): 2.2.1+cu121 (False)
Tensorflow version (GPU?): 2.16.1 (False)
Flax version (CPU?/GPU?/TPU?): not installed (NA)
Jax version: not installed
JaxLib version: not installed
Using GPU in script?: no
Using distributed or parallel set-up in script?: no

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Use the latest tensorflow native version (2.16.1) and latest transformers version (4.39.1)
2.  I am trying to use the evaluator() from evaluate package to evaluate and compute to compute the scores
3.  The dataset on which I am performing the validation is "multinews" and the model is https://huggingface.co/KostiuchenkoArtem/bart_large_multi_modified.
4. I am seeing variations from the 4th decimal in the rouge scores.

### Expected behavior

Same rouge score everytime

Comments:
Fourth decimal is 1e-4, this seems acceptable for such an old model given different settings / torch versions no? 

I don't change any settings between the runs and what I ideally expect is the same scores, as the dataset and the model weights don't change. So I had a doubt if this variation is expected for rouge scores or not

Again for a very old model, this should be expected

Ok thank you for your response.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
`generate` method does not work with `use_cache=False` in `Llama-2` model with `model.config._attn_implementation = "eager"`
Author: nurlanov-zh
Pull request: no
Labels: name: Generation

name: Cache

@ArthurZucker , @gante 

The issue still persists when `model.config._attn_implementation = "eager"` is used. Currently it works because the implementation relies on `sdpa` attention implementation, which does not use `causal_mask`. 
```py
class LlamaModel(LlamaPreTrainedModel):
    ...

    def _update_causal_mask(
        self,
        attention_mask: torch.Tensor,
        input_tensor: torch.Tensor,
        cache_position: torch.Tensor,
        past_seen_tokens: int,
    ):
        ...

        if self.config._attn_implementation == "sdpa":
            # For SDPA, when possible, we will rely on its `is_causal` argument instead of its `attn_mask` argument,
            # in order to dispatch on Flash Attention 2.
            if AttentionMaskConverter._ignore_causal_mask_sdpa(
                attention_mask, inputs_embeds=input_tensor, past_key_values_length=past_seen_tokens
            ):
                return None
```

Ideally, the code should work with any attention implementation. I believe the problem is that the `cache_position` is wrong when we use `model.generate` with `use_cache=False`.

_Originally posted by @nurlanov-zh in https://github.com/huggingface/transformers/issues/30347#issuecomment-2076917488_
            

Comments:
Please provide a reproducer 😉 

Reproduction:

```py
from transformers import AutoTokenizer, LlamaForCausalLM
import torch


model_path = "meta-llama/Llama-2-7b-chat-hf"
device = "cuda:0"
torch_dtype = torch.float16

model = LlamaForCausalLM.from_pretrained(model_path, torch_dtype=torch_dtype).to(device)
tokenizer = AutoTokenizer.from_pretrained(model_path)

model.config._attn_implementation = "eager"
model.config.use_cache = False
model.generation_config.use_cache = False

prompt = """
 
 
 There's a llama in my garden. What should I do? 
"""
input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generate_ids = model.generate(input_ids, max_length=300)
output = tokenizer.batch_decode(generate_ids, skip_special_tokens=False, clean_up_tokenization_spaces=False)[0]
print(output)
```

I have a PR that fixes this bug: #30485
@ArthurZucker , could you take a look?

Sure!
Fix `cache_position` initialisation for generation with `use_cache=False`
Author: nurlanov-zh
Pull request: yes

# What does this PR do?
Fixes #30482


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker @younesbelkada @gante

Comments:
Can you just rebase on main? 🤗 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30485). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Can you just rebase on main? 🤗

@ArthurZucker , done ✔️


Thanks 🤗
Updated docs of `forward` in `Idefics2ForConditionalGeneration` with correct `ignore_index` value
Author: zafstojano
Pull request: yes



# What does this PR do?

The docs in the `forward` method of the `Idefics2ForConditionalGeneration` incorrectly states that `-100` is the value in the `labels` variable for which the `CrossEntropyLoss` will ignore the appropriate indices:
https://github.com/huggingface/transformers/blob/277db238b70c5c8bb5a445b75becbfd79644db42/src/transformers/models/idefics2/modeling_idefics2.py#L1777-L1780

However, it can be seen in this line that in fact the model uses the `image_token_id` as an ignore index:

https://github.com/huggingface/transformers/blob/277db238b70c5c8bb5a445b75becbfd79644db42/src/transformers/models/idefics2/modeling_idefics2.py#L1860

This PR updates the docs with the correct information.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Ping @amyeroberts 


Comments:
Separate tokenizer tests
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Make sure we run the tokenization specific tests in this workflow. Should allow better splitting.
This reduces the `iddle` time of the torch/tf/flax jobs. Improving further the time ci take to run.

In this PR I make sure that only the _tf_ _flax_ test files are passed to the corresponding tests, but this might not fair well with testing that the tests are protected WDYT? 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30675). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
How to All Utilize all GPU's when device="balanced_low_0" in GPU setting
Author: kmukeshreddy
Pull request: no

### System Info

I know that while loading the model in "balanced_low_0" GPU setting the model is loaded into all GPU's apart from 0: GPU. Where the 0: GPU is left to do the text inference. (i.e. text inference as in performing all the calculation to generate response inside the LLM)

So, as per the give device parameter my model is loaded onto 1,2,3 GPU's and 0: GPU is left for inference.

| ID | GPU | MEM |
| 0 | 0% | 3% |
| 1 | 0% | 83% |
| 2 | 0% | 82% |
| 3 | 0% | 76% |

Question: How can i also utilize the remaining 1,2,3 GPU's to perform text inference not only 0:GPU?

Context: "balanced_low_0" evenly splits the model on all GPUs except the first one, and only puts on GPU 0 what does not fit on the others. This option is great when you need to use GPU 0 for some processing of the outputs, like when using the generate function for Transformers models

Reference: https://huggingface.co/docs/accelerate/en/concept_guides/big_model_inference#designing-a-device-map

CC: 
@gante @ArthurZucker  and @younesbelkada 

Apologies if the ticket is raised under different bucket

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

na

### Expected behavior

na

Comments:
Closing this issue, as opened this issue at right position.

Reopening the issue.

Hey 🤗  thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
In this case, you are gonna use gpu 1, 2, 3 for inference, it's just automatically done under the hood by accelerate
Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Thank you!!
Restructure
Author: aymeric-roucher
Pull request: yes
Labels: name: Agents

# What does this PR do?

- Rename `tools` folder to `agents`, refactor it
- Fix style tests
- Some small tweaks

Comments:
@aymeric-roucher Why is it closed? These tools are still not working: https://huggingface.co/collections/huggingface-tools/diffusion-tools-6630bb19a942c2306a2cdb6f. 
```from transformers.agents.tools import launch_gradio_demo
ModuleNotFoundError: No module named 'transformers.agents'
Update `workflow_id` in `utils/get_previous_daily_ci.py`
Author: ydshieh
Pull request: yes

# What does this PR do?

After #30313, we still don't get the missing new model failure report on slack channel. 

It is because we changed the daily ci and the main workflow file to trigger is a new file (`.github/workflows/self-scheduled-caller.yml`, added in #30012). This change to `workflow_id` which is used in `utils/get_previous_daily_ci.py`. The `new model failure report` relies on this file, and since the `workflow_id` is no longer the correct one, it is not produced.

This PR updates the `workflow_id`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30695). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.