noamaanMulla-03/transformers-issues huggingface dataset:
Make style failed
Author: jiqing-feng
Pull request: no

### System Info

No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and rerun for more info.)

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.36.0.dev0
- Platform: Linux-5.16.0-rc8-intel-next-01534-g53cb5f883cf7-x86_64-with-glibc2.28
- Python version: 3.9.16
- Huggingface_hub version: 0.19.4
- Safetensors version: 0.3.1
- Accelerate version: 0.20.3
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): 0.7.4 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When I run `make style`,  I got the following error

```
ruff check examples tests src utils setup.py conftest.py --fix
error: TOML parse error at line 17, column 1
   |
17 | [tool.ruff.format]
   | ^^^^^^^^^^^^^^^^^^
wanted exactly 1 element, more than 1 element

make: *** [Makefile:67: style] Error 2
```

### Expected behavior

It should be reformat all codes.

Comments:
Hi @jiqing-feng, thanks for raising this issue! 

A recent PR #27144 updated our formatting logic to use ruff formatter instead of black. Could you try updating ruff and run again? `pip install -U ruff` 

Had the same issue & updating ruff solved this for me

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
save_total_limit incorrectly deletes checkpoints
Author: huchinlp
Pull request: no

### System Info

transformers==4.34.1

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I set `save_total_limit` to 10, but surprisingly found it deleting my **latest** model. It was supposed to keep checkpoint-10000 and subsequent checkpoints, but instead, I found that the following checkpoints remained:

```checkpoint-9000
checkpoint-9100
checkpoint-9200
checkpoint-9300
checkpoint-9400
checkpoint-9500
checkpoint-9600
checkpoint-9700
checkpoint-9800
checkpoint-9900
```

I suspect that this may be due to the deletion process using the string names of the files rather than their numerical value.

### Expected behavior

It should keep the latest checkpoints.

Comments:
FYI in case this is relevant:
- https://github.com/huggingface/transformers/issues/26961

cc @muellerz @pacman100 

Hello, please refer to https://github.com/huggingface/transformers/issues/26961#issuecomment-1827691624

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
'LlamaTokenizerFast' object has no attribute 'prefix_id'
Author: Kushalamummigatti
Pull request: no

### System Info

Using transformer version  `transformers` version: 4.34.1
- Platform: Linux-5.10.0-26-cloud-amd64-x86_64-with-glibc2.31
- Python version: 3.10.10
- Huggingface_hub version: 0.19.4
- Safetensors version: 0.3.3
- Accelerate version: 0.23.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
@ArthurZucker 



### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Am trying to use codellama tokenization function present in llama/generation.py for a custom problem. But am facing this issue: 'LlamaTokenizerFast' object has no attribute 'prefix_id'

Please help. 
def infilling_prompt_tokens(
    tokenizer: Tokenizer,
    pre: str,
    suf: str,
    suffix_first: bool = False,
) -> List[int]:
    """
    Format and encode an infilling problem.
    If `suffix_first` is set, format in suffix-prefix-middle format.
    """
    assert tokenizer.prefix_id is not None
    assert tokenizer.middle_id is not None
    assert tokenizer.suffix_id is not None
    if suffix_first:
        # format as "<PRE> <SUF>{suf} <MID> {pre}"
        return (
            [tokenizer.bos_id, tokenizer.prefix_id, tokenizer.suffix_id]
            + tokenizer.encode_infilling(suf)
            + [tokenizer.middle_id]
            + tokenizer.encode(pre, bos=False, eos=False)
        )
    else:
        # format as "<PRE> {pre} <SUF>{suf} <MID>"
        return (
            [tokenizer.bos_id, tokenizer.prefix_id]
            + tokenizer.encode(pre, bos=False, eos=False)
            + [tokenizer.suffix_id]
            + tokenizer.encode_infilling(suf)
            + [tokenizer.middle_id]
        )

### Expected behavior

The function should be able to reproduce on any custom format. 

Comments:
Hello, 🤗 from the look of it, you are not using the `CodeLlamaTokenizer` but the `LlamaTokenizer`. You code does not include a reproducer so I cannot really help you here. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
type object 'OPTDecoder' has no attribute '_prepare_decoder_attention_mask'.
Author: muzi0111
Pull request: no


I'm currently facing an issue as follows: which version of Transformers should I install? The latest version does not seem to have this configuration option. I'm getting an AttributeError: type object 'OPTDecoder' has no attribute '_prepare_decoder_attention_mask'.

Comments:
Hey! I think that version 4.34.0 should work. This attribute was removed in #27086 and it was a private method so not a breaking change

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Update docs around mixing hf scheduler with deepspeed optimizer
Author: dwyatte
Pull request: yes

# What does this PR do?

Looks like the docs around mixing and matching deepspeed optimizers with HF schedulers were not updated with https://github.com/huggingface/transformers/pull/25863. I think there's probably an analogous update to make in [`accelerate`](https://github.com/huggingface/accelerate/blob/ceb7c699bc36bdb3bbf32cceaaca2d1ceaf62dae/docs/source/usage_guides/deepspeed.md?plain=1#L418-L419)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@muellerzr, @pacman100

Comments:
Fix trainer saving safetensors: metadata is None
Author: hiyouga
Pull request: yes

# What does this PR do?

Fixes https://github.com/hiyouga/LLaMA-Factory/issues/1959

If we use `Trainer` to train a model that does not belong to the `PreTrainedModel` class, such as the `PreTrainedModelwithValuehead` from the TRL library, the trainer will not save the metadata. This leads to errors in reading the metadata when using `AutoModelforCausalLM.from_pretrained` to load the model.

https://github.com/huggingface/transformers/blob/29e7a1e1834f331a4916853ecd58549ed78235d6/src/transformers/trainer.py#L2911

https://github.com/huggingface/transformers/blob/29e7a1e1834f331a4916853ecd58549ed78235d6/src/transformers/modeling_utils.py#L3403-L3407


Although it may sound strange to load a model that does not belong to the `PreTrainedModel` class using `AutoModelForCausalLM.from_pretrained`, this approach benefits model loading by utilizing features such as low_cpu_mem_usage if the model checkpoints share the same structure.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@muellerzr @pacman100


Comments:
Pix2StructForConditionalGeneration not exported
Author: mesmerlord
Pull request: no

### System Info

transformers version = 4.27.4
torch = 2.0.0

Getting an import error

ImportError: cannot import name 'Pix2StructForConditionalGeneration' from 'transformers' (D:\code\alpaca\env\lib\site-packages\transformers\__init__.py)

And when I check the package files, I don't see pix2struct in transformers/models
![image](https://user-images.githubusercontent.com/76161333/228655699-b9f32fcc-ce53-42a1-a16a-ea9adbe5a3c8.png)

This is how it is in git source code : 
![image](https://user-images.githubusercontent.com/76161333/228656078-b3b39601-8e9c-4916-b82b-d24c608d6300.png)


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import (
    Pix2StructForConditionalGeneration,
    Pix2StructProcessor,
)

### Expected behavior

Imports occur successfully.

Comments:
Yes, Pix2Struct was not in the last release. You need to install `transformers` [from source](https://huggingface.co/docs/transformers/installation#install-from-source) to use it.

For anyone else getting `ImportError: cannot import name 'Pix2StructForConditionalGeneration' from 'transformers'`:
This was added in https://github.com/huggingface/transformers/commit/0f68a7f4088cac3a97b28e339d8b8d72c0edf93b
so you need `transformers` >= 4.28.0
[`Whisper`] nit
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Was getting these strange warnings:
```python
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
Ignored unknown kwarg option normalize
```
with processor = AutoProcessor.from_pretrained("asapp/sew-d-tiny-100k-ft-ls100h") 

Comments:
@ArthurZucker is there any plans to include this fix in patch release?

I don't think so! But the next release will have it
[fix] Optimize deletion speed while truncate_sequences (#3563)
Author: liyuqing1
Pull request: yes

# What does this PR do?
The process remains unchanged, but the writing method is changed to improve the speed of truncate_sequences().
"del ids[-1]" is an in-place operation, while the original "ids = ids[:-1]"  make an additional copy of ids[0:-1]. Therefore, the original writting method is slower, especially when sentences are long.



## Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker and @younesbelkada

Comments:
Sorry, this method may be related to the environment and len(ids), so the rewriting may not always be effective. We are conducting further research to better address the issue.

No worries feel free to ping me if you have good findings!
Remove fast tokenization warning in Data Collators
Author: dbuos
Pull request: yes

# What does this PR do?

Modifies the data collators (DataCollatorWithPadding, DataCollatorForTokenClassification, DataCollatorForSeq2Seq, ...) to pad without warning the user to instead calling tokenizer.__call__ when using fast tokenizers.

The modification in the Data collators:
- we save the state of the tokenizer with regards to the warning
- disable the warning
- pad
- restore the state of whether we want to warn or not.

See #22638

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #19471 and  #22638


## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@sgugger

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @


 -->


Comments:
@amyeroberts
[Efficiency] Decoding can be made faster by not converting special tokens to ids for each token.
Author: ganeshpatelQB
Pull request: no
Labels: name: WIP

### System Info

- `transformers` version: 4.29.0.dev0
- Platform: macOS-14.0-arm64-arm-64bit
- Python version: 3.11.4
- Huggingface_hub version: 0.13.3
- Safetensors version: not installed
- PyTorch version (GPU?): 2.0.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

The following function is being called for each token while using decoding function.

```python

from transformers import T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained(TOKENIZER_PATH)
beams = tokenizer.batch_decode(
    outputs, skip_special_tokens=True
)
```

```python
  @property
  def all_special_ids(self) -> List[int]:
      """
      `List[int]`: List the ids of the special tokens(`'<unk>'`, `'<cls>'`, etc.) mapped to class attributes.
      """
      all_toks = self.all_special_tokens
      all_ids = self.convert_tokens_to_ids(all_toks)
      return all_ids
```



### Expected behavior

all_special_ids should not be called for each token while decoding at the time of inferencing.

Comments:
Very good catch! I'll open a pr for this. Affect both `convert_ids_to_tokens` and decode. 🤗 I need to do some benchmarking as I suspect this does won't have a huge impact but will give it a shot. I plan to benchmark our full calls to make sure we don't have things similar to this else where

My initial tests did not show any impact with NLLB and whisper which have the most amount of added tokens, but I'll try to optimize and benchmark in a near futur!
max_new_tokens is set， but longer output is generated
Author: yuqie
Pull request: no

Hi, I use llama2-7b and set `max_new_tokens=1024`, when I test with several samples, some output larger than 1024 are obtained. I have no idea with this, could anyone can help me.

```
llm = AutoModelForCausalLM.from_pretrained(model, torch_dtype=torch.float16, trust_remote_code=trust_remote_code)
tokenizer.pad_token = tokenizer.eos_token
llm = llm.cuda()

output_num_tokens = []
for i in tqdm(range(len(requests))):
        prompt = requests[i]
        # Generate the sequences.
        input_ids = tokenizer(prompt, return_tensors="pt",padding=True).input_ids
        llm_outputs = llm.generate(
            input_ids=input_ids.cuda(),
            do_sample=not use_beam_search,
            num_return_sequences=1,
            temperature=1.0,
            top_p=1.0,
            use_cache=True,
            max_new_tokens=1024,
        )
        tokenizer.decode(llm_outputs[0], skip_special_tokens=True)
        output_num_tokens.append(len(llm_outputs[0]))

print(output_num_tokens)
```
the results are as following:
```
 [1062, 664, 219, 553, 1042, 1001, 313, 1128, 584, 984, 330, 710, 1415, 1429, 1526, 362, 906, 642, 1036, 207, 111, 394, 561, 672, 407, 223, 849, 869, 1675, 1170]
```

Comments:
You are not setting the correct parameter, what you are looking for is `max_length` 🤗 

Hi @ArthurZucker , just a doubt....in the documentation, it is written that they both serve the same purpose or am i missing something. Would be happy to be corrected.
Should I approach the forum for this doubt?
<img width="443" alt="image" src="https://github.com/huggingface/transformers/assets/56391795/fe178fa0-2f0e-4423-90ff-4f6aa9a18fe9">


> You are not setting the correct parameter, what you are looking for is `max_length` 🤗

`max_length` works well, thx. 
As @drunkeninja42 mentioned above, I am also confused with the parameters  `max_length`  and `max_new_tokens`, what the difference and when should `max_new_tokens` be used？

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

`max_length` means input_length + generated_tokens < max_length, while max_new_tokens mean only generated_tokens < max_new_tokens
Conv1d would be initialized to all zeros in a pretrained_model
Author: Hannibal046
Pull request: no

### System Info

- `transformers` version: 4.36.2
- Platform: Linux-5.15.0-1050-azure-x86_64-with-glibc2.17
- Python version: 3.8.18
- Huggingface_hub version: 0.20.1
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import BertModel
import torch
class MyBERT(BertModel):
    def __init__(self,config):
        super().__init__(config)
        self.conv = torch.nn.Conv1d(
            in_channels=768,
            out_channels=768,
            kernel_size = 16,
            )
        self.post_init()
mybert = MyBERT.from_pretrained("bert-base-uncased")
print(mybert.conv.weight.data[0])
```

![image](https://github.com/huggingface/transformers/assets/38466901/584e2717-fd9a-48fe-af69-02f2c40d67cc)
![image](https://github.com/huggingface/transformers/assets/38466901/5976396a-0852-4a2a-a501-6bed59b6d64a)


### Expected behavior

The conv layer would be initialized properly.

Comments:
It seems that the problem was in `from_pretrained`. When I init the model with config, it works fine.

![image](https://github.com/huggingface/transformers/assets/38466901/c1324715-9e6f-4b2c-bdab-fc32a50a3e9a)


OK, I see. It is a feature for LLM loading. Using the following would solve the problem:
```python
MyBERT.from_pretrained("bert-base-uncased",_fast_init=False)
```

Hey! It's more like[ the init scheme](https://github.com/huggingface/transformers/blob/b1292bca6923cfbc9cb3f70cb55df57e4e17e630/src/transformers/models/bert/modeling_bert.py#L744)

needs to be updated to your needs! `_fast_init` will skip normal inits and `_init_weights` is the only function that will be called for this. 

Got it! Appreciate