noamaanMulla-03/transformers-issues huggingface dataset:
Use proper weights name when logging in `PreTrainedModel.save_pretrained`
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

It seems that `WEIGHTS_NAME` even when `safe_serialization=True`. This PR fixes that by using `weights_name` which is defined as follows:

```
  if not _hf_peft_config_loaded:        
      weights_name = SAFE_WEIGHTS_NAME if safe_serialization else WEIGHTS_NAME        
      weights_name = _add_variant(weights_name, variant)        
  else:        
      weights_name = ADAPTER_SAFE_WEIGHTS_NAME if safe_serialization else ADAPTER_WEIGHTS_NAME
```

[Link to the code](https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_utils.py#L2189-L2193).

Comments:
Could you rebase and make sure CIs are green? 🤗 

I think the issue is fixed in the current `main` branch now.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27731). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Generate: deprecate old public functions
Author: gante
Pull request: yes

# What does this PR do?

Schedules for deprecation old public functions -- these functions are not used anywhere in the code base, and haven't been since I've been in charge of `generate`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28478). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Docs: add model paths
Author: gante
Pull request: yes

# What does this PR do?

As reported by @sayakpaul: some models had placeholder paths. This PR corrects it.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28475). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
LlamaForCausalLM does not support Flash Attention 2.0 yet
Author: Patrick-Ni
Pull request: no

The model was loaded with use_flash_attention_2=True, which is deprecated and may be removed in a future release. Please use `attn_implementation="flash_attention_2"` instead.
Traceback (most recent call last):
  File "/root/paddlejob/workspace/env_run/benchmark/generation/main.py", line 116, in <module>
    main()
  File "/root/paddlejob/workspace/env_run/benchmark/generation/main.py", line 91, in main
    pipeline = load_model_and_tokenizer(model_home, args.model, args.use_pipeline)
  File "/root/paddlejob/workspace/env_run/benchmark/generation/load_models_and_datasets.py", line 26, in load_model_and_tokenizer
    model = AutoModelForCausalLM.from_pretrained(
  File "/root/paddlejob/workspace/env_run/lib/python3.9/site-packages/transformers/models/auto/auto_factory.py", line 561, in from_pretrained
    return model_class.from_pretrained(
  File "/root/paddlejob/workspace/env_run/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3456, in from_pretrained
    config = cls._autoset_attn_implementation(
  File "/root/paddlejob/workspace/env_run/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1302, in _autoset_attn_implementation
    cls._check_and_enable_flash_attn_2(
  File "/root/paddlejob/workspace/env_run/lib/python3.9/site-packages/transformers/modeling_utils.py", line 1382, in _check_and_enable_flash_attn_2
    raise ValueError(
ValueError: LlamaForCausalLM does not support Flash Attention 2.0 yet. Please open an issue on GitHub to request support for this architecture: https://github.com/huggingface/transformers/issues/new

Comments:
make sure you are using the latest version of transformers and not a code on the hub! 🤗 closing as this was added in #25598
Generate: refuse to save bad generation config files
Author: gante
Pull request: yes

# What does this PR do?

This PR converts a warning into an exception. This warning stated that it would be converted to an exception in v4.34

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28477). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
TF: purge `TFTrainer`
Author: gante
Pull request: yes

# What does this PR do?

Removes `TFTrainer` and all its traces.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28483). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
chore: Just fix some typo
Author: hugo-syn
Pull request: yes

# What does this PR do?

Just fix some typo


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Hi @hugo-syn thanks for opening a PR! There doesn't seem to be any changes in this PR?  

Indeed, I don't understand. I've synchronized the main repo and my changes don't appear here, but I can see them on my fork. 

The typo fixes in b30d46622e7b3f839fb93cde269dfc797319979b have already been applied to the repo, in a previous PR last week #28361 by you.

Closing then sorry for my mistake :)
[ASR Pipe] Fix num frames for bs > 1
Author: sanchit-gandhi
Pull request: yes

# What does this PR do?

Handles the case when the pipeline is batched and we want to return word-level timestamps. Here, we have a batch (list) of strides, which we slice to get the first stride value.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26699). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Any plan for a fix for this?

Reopened this @sanchit-gandhi! (not sure if we still want to work on this, if not, then feel free to close)

This would enable word level timestamps for whisper with batched inference right? Would love to see this, or another workaround if that's possible 

Yes! @thomasmol 🤗 

This would be super valuable to so many applications that require word level transcriptions

Any updates on this feature? It would be a massive advantage

On testing via T4 GPU, while this PR does fix the issue, the GPU consumption ramps up quite significantly.
While processing a 3 min 19 audio file, without word timestamps, it is able to complete the process with 24 batch size (default) under 30 seconds, with GPU consumption ~9.5 GiB.
But, with the word timestamps, it causes GPU OOM, with 24 batch size, the best I was able to do with 16 GB T4 GPU was batch size as 2, which took around 2.5 minutes, while still GPU consumption with this batch size going to around ~10 GiB.

We need this to be fixed for insanely fast whisper, is there any eta on the pr?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

#28114 probably addressed this 

Closed by https://github.com/huggingface/transformers/pull/28114.
Fix docstrings and update docstring checker error message
Author: Rocketknight1
Pull request: yes

While I was making fixes to the docstring checker, I found another issue - this one seems to be intermittent, and I'm not sure why it only fails tests sometimes. Still, it's definitely wrong, so this fix should hopefully avoid issues in future!

Comments:
Okay there's something more deeply wrong with the docstring checker than I realized - the new docstring is definitely more correct, but is failing checks in the CI! Will investigate.

Quick re-review request @ArthurZucker, I'm going to update the error message in this PR too!
Change progress logging to once across all nodes
Author: siddartha-RE
Pull request: yes

# What does this PR do?

Change progress logging to be once across all nodes rather than once per node.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Library:
- trainer: @muellerzr and @pacman100


Comments:
> Let's just include the changes to `trainer_callback.py` please, and #28364 we'll leave for handling the checkpoint issue. Thanks!

Done. Thank you for reviewing.

@muellerzr What do you think? Will close if you think this does not seem worth changing..
Fix load correct tokenizer in Mixtral model documentation
Author: JuanFKurucz
Pull request: yes

# What does this PR do?

There is an incorrect non existent tokenizer linked in the Mixtral documentation. https://huggingface.co/docs/transformers/main/en/model_doc/mixtral#usage-tips

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Models:

- text models: @ArthurZucker and @younesbelkada

Documentation: @stevhliu and @MKhalusova


Comments:
Roundtrip Tokenization Failure
Author: YehowshuaScaled
Pull request: no

### System Info

- `transformers` version: 4.36.0.dev0
- Platform: Linux-6.2.0-37-generic-x86_64-with-glibc2.37
- Python version: 3.11.4
- Huggingface_hub version: 0.19.4
- Safetensors version: 0.4.0
- Accelerate version: 0.24.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.0.dev20231126+rocm5.7 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

run the following:

```python3
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained(
    "mistralai/Mistral-7B-v0.1",
    add_eos_token=False,
    add_bos_token=False)

pre_roundtrip_seq = [523, 28713, 28767]

print(f"{pre_roundtrip_seq=}")
print(f"{tokenizer.decode(pre_roundtrip_seq)=}")

post_roundtrip_seq = tokenizer.encode(tokenizer.decode(pre_roundtrip_seq))

print(f"{post_roundtrip_seq=}")
print(f"{tokenizer.decode(post_roundtrip_seq)=}")
```

```python3 bug.py
pre_roundtrip_seq=[523, 28713, 28767]
tokenizer.decode(pre_roundtrip_seq)='&lt;s&gt;'
post_roundtrip_seq=[1]
tokenizer.decode(post_roundtrip_seq)='&lt;s&gt;'
```
### Expected behavior

It's quite conceivable and sometimes desirable that a language model could spit out the literal sequence "&lt;s&gt;" which is a less than followed by s followed by greater than as shown above.

As is, tokenizers would interprets this as a start token. This is clearly incorrect behavior. To get around this behavior, I've been manually instantiating Tokenizers from `SentencePiece` and passing them a model binary.

Clearly, by doing this, I can longer take advantage of much of the Transformer library including generate or Trainer, as both expect a `Transformer.Tokenizer`.

I'm not sure what a clean way to get around this is - could involve re-doing much of the `Transformer.Tokenizer` infrastructure - unless there is some special token escape sequence included with `Transformer.Tokenizer` that I'm simply not aware of.

Comments:
As an example, &lt;s&gt; could show up when having a model generate or classify HTML. I think this is a very valid use case.

The slow tokenizer support the `encode_special_tokens` option which is set to false by default because most common use case need the special token to be recognized. Fast is going to support that in the near futur! 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Predicted Label to Predicted token mapping in LayoutLMV3. Please look at the description.
Author: slk-genai
Pull request: no

### Feature request

I am trying the below code:

processor = LayoutLMv3Processor.from_pretrained("/content/layoutlmv3-large-finetuned-funsd")
encoding = processor(image, return_offsets_mapping=True, return_tensors="pt")
offset_mapping = encoding.pop('offset_mapping')
print(encoding.keys())
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
for k,v in encoding.items():
  encoding[k] = v.to(device)
model = LayoutLMv3ForTokenClassification.from_pretrained("/content/layoutlmv3-large-finetuned-funsd")
model.to(device)
outputs = model(**encoding)
print(outputs.logits.shape)

I can see that there are parts of image which are getting labelled but during OCR identification those texts are not getting identified. 
So, how can I get the label and their respective token?

### Motivation

I am new to LayoutLMv3 and I have task in hand which was getting easily done but I can't find a way to map the token prediction and it's respective label prediction mapping. I am not using the above checkpoints but the problem is similar to the mentioned checkpoint. 

### Your contribution

Let me know if you need any help.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey 🤗  thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co/) instead? I'm sure the community will be of help!

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
optimizer.param_groups[0]['params'][0].dtype IndexError: list index out of range
Author: wandouguo
Pull request: no

### System Info

errors happened:
deepspeed                 0.12.3
transformers              4.35.2  

normal:
transformers              4.33.2
deepspeed                 0.12.3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1、
deepspeed                 0.12.3
transformers              4.35.2  
2、{
    "fp16": {
        "enabled": "auto",
        "loss_scale": 0,
        "loss_scale_window": 100,
        "initial_scale_power": 16,
        "hysteresis": 2,
        "min_loss_scale": 1e-10
    },
    "zero_optimization": {
        "stage": 2,
        "allgather_partitions": true,
        "allgather_bucket_size": 1e8,
        "overlap_comm": true,
        "reduce_scatter": true,
        "reduce_bucket_size": 1e8,
        "contiguous_gradients": true
    },
    "gradient_accumulation_steps": "auto",
    "gradient_clipping": "auto",
    "steps_per_print": 2000,
    "train_batch_size": "auto",
    "train_micro_batch_size_per_gpu": "auto",
    "wall_clock_breakdown": false
}

3、trainer.train() 

### Expected behavior

normal training

Comments:
Hello, if you want our help we are going to need a full reproducer (as the contribution guidelines point out)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`chore`] Update warning text, a word was missing
Author: tomaarsen
Pull request: yes

Hello!

# What does this PR do?

* Updates a warning text: "lead" was missing.

I missed this in the original PR, apologies.

## Before submitting
- [x] This PR fixes a typo or improves the docs

## Who can review?

@ArthurZucker

cc: @stas00 Thanks for pointing this out here: https://github.com/huggingface/transformers/pull/26681#discussion_r1425760204

- Tom Aarsen

Comments:
@amyeroberts This should be a simple merge; but I'd like to leave it to a core maintainer as I'm not 100% sure whether non-core maintainers should be merging on `transformers`.

We usually let HF people merge their own PRs 😉 Could you rebase on main before merging? 

Good to know!

Rebased & force pushed :) 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28017). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[`core`/ FEAT] Add the possibility to push custom tags using `PreTrainedModel` itself
Author: younesbelkada
Pull request: yes

# What does this PR do?

From an idea we discussed internally. This PR introduces a new API to inject custom tags in the model card

From a community perspective it will make easier to push custom tags as for now it is only limited to trainers. 

Below I demonstrate the simplicity of the API

```python
from transformers import AutoModelForCausalLM

model_name = "HuggingFaceM4/tiny-random-LlamaForCausalLM"
model = AutoModelForCausalLM.from_pretrained(model_name)

model.add_model_tags(["tag-test"])
model.push_to_hub("llama-tagged")
```

cc @osanseviero @Narsil @julien-c 

Note with the current design each time a user calls `push_to_hub`, it will create a model card template if no model card is present on the hub

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28405). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I made some modification after some offline discussion, an exmaple model pushed can be found here: https://huggingface.co/ybelkada/test-tags-model lmk wdyt @amyeroberts @ArthurZucker @Wauplin 
I also updated the API on the PR description

@amyeroberts @Wauplin I think that I have addressed most of your comments now, the API is much more flexible and less "agressive" thanks to your suggestions
1- Creates a simple description
2- Moved `transformers` to `library-name`
3- Now Trainer and `model_tags` are compatible together, which was not the case in the first commits. e.g. when combining `SFTTrainer` from trl that automatically pushes `sft` and `trl` tags and this PR, all tags gets successfully pushed. E.g.:
```python
from transformers import AutoModelForCausalLM, TrainingArguments
from trl import SFTTrainer
from datasets import load_dataset

model_name = "HuggingFaceM4/tiny-random-LlamaForCausalLM"
model = AutoModelForCausalLM.from_pretrained(model_name)

model.add_model_tags(["tag-test"])

train_data = load_dataset("imdb", split="train")

trainer = SFTTrainer(
    model=model,
    args=TrainingArguments(output_dir="test-tag-already-tagged"),
    train_dataset=train_data,
    dataset_text_field="text"
)

trainer.push_to_hub("ybelkada/test-tag-already-tagged")
```
The example repo is here: https://huggingface.co/ybelkada/test-tag-already-tagged
4- If tags already exists it will not overwrite all the tags but add the new tags with existing tags. e.g. if one does `model.add_tags(["test-tag-2"])` after already pushing a model with a tag to Hub, as you can see from this repository: https://huggingface.co/ybelkada/test-model-already-tagged the new tag gets appended

cc @osanseviero as well

Thanks for handling this @younesbelkada