noamaanMulla-03/transformers-issues huggingface dataset:
Replace `accelerator.use_fp16` in examples
Author: hlky
Pull request: yes

# What does this PR do?

`examples_torch` tests are currently failing due to usage of `accelerator.use_fp16`, `use_fp16` was removed in [accelerate#3098](https://github.com/huggingface/accelerate/pull/3098) and `transformers` CI uses accelerate@main so the property is no longer available.

This PR replaces `accelerator.use_fp16` with `accelerator.mixed_precision != "no"` which is equivalent to the removed `use_fp16` property. 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

cc @muellerzr


Comments:
If we are using fp8, we need to pad to 16. Since these are simple examples + did not take into account fp8 from the start, I guess we can leave it like that. Otherwise, we would have to do the following

```py
  if accelerator.mixed_precision == "fp8":
            pad_to_multiple_of = 16
        elif accelerator.mixed_precision != "no":
            pad_to_multiple_of = 8
        else:
            pad_to_multiple_of = None
```

Thanks for the review, I've gone ahead and added `pad_to_multiple_of = 16` for fp8.

Looks like `mixed_precision` was introduced a while ago in [accelerate#247](https://github.com/huggingface/accelerate/pull/247), the now removed `use_fp16` property was changed to use `mixed_precision` at the same time.

> Looks like mixed_precision was introduced a while ago in https://github.com/huggingface/accelerate/pull/247, the now removed use_fp16 property was changed to use mixed_precision at the same time.

Thanks for looking into ! Looks like this was done in 2022, so with <= accelerate v.0.20. The min version of accelerate in transformers is v0.26.0.

Merging this since this should be safe to do 

Thanks both for checking!
Generate: assistant should sample when the main model samples
Author: gante
Pull request: yes

# What does this PR do?

Fixes #32867 

TL;DR in assisted generation, the assistant model must sample when the main model is sampling. Otherwise, mathematical properties in the corresponding code path do not hold (see speculative decoding paper).

This reverts #30778, where I forced the assistant model to always run greedy decoding for speed purposes (more matched candidate tokens = faster).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33534). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
RecurrentGemma: add generative tests
Author: gante
Pull request: yes

# What does this PR do?

Adds generative tests to `RecurrentGemma`, through the help of the `_is_stateful` flag (see #31478). A few changes were needed to handle failing tests, but we could remove many overwrites 🤗 

⚠️ `position_ids` was not being passed into the model at generation time, padding was broken



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31498). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker pushed further corrections. As per your last set of comments, the current diff doesn't break `torch.compile` on recurrent gemma  :)
Fix some missing tests in circleci
Author: ydshieh
Pull request: yes

# What does this PR do?

Run tests in `tests/generation` etc. in a new job

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33559). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hi @gante !

The PR #33219 has a change

```
        # Special case: some models have generation attributes set in the decoder. Use them if still unset in the
        # generation config (which in turn is defined from the outer attributes of model config).
        decoder_config = model_config.get_text_config(decoder=True)
```
which cause 

```bash
        config_dict = model_config.to_dict()
        # generation config (which in turn is defined from the outer attributes of model config).
>       decoder_config = model_config.get_text_config(decoder=True)
E       AttributeError: 'GenerationConfig' object has no attribute 'get_text_config'

src/transformers/generation/configuration_utils.py:1240: AttributeError
================================================================================ short test summary info =================================================================================FAILED tests/generation/test_configuration_utils.py::GenerationConfigTest::test_initialize_new_kwargs - AttributeError: 'GenerationConfig' object has no attribute 'get_text_config' 
```

Could you take a look 🙏 ?

Somehow a bit strange `model_config` is a `GenerationConfig` ... 🤔
Update daily ci to use new cluster
Author: ydshieh
Pull request: yes

# What does this PR do?

And re-enable daily (which was disabled in #33136)

I haven't had a final confirmation from @glegendre01 , but let's take a chance to run it during this weekend.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33627). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Thanks for updating!
> 
> Only comment is that the mapping logic from the aws cluster name to "x-gpu" is very repetitive. I don't know of an obvious way to abstract this out in the yaml, but it would be cool if we could :)

Yes I agree. I will think of it but let's try to run the CI during this weekend and potentially give feedbacks to the infra team on Monday 🙏 Thank you for the review.
Fix typos
Author: litianjian
Pull request: yes

# What does this PR do?
This PR fixes a typo. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->




Comments:
Adding an `Open in DevZero` button
Author: eirol1221
Pull request: yes

# What does this PR do?

DevZero is a dev environment platform. Using this link, all contributors to this project will be able to use a DevZero environment as their dev workspace.
DevZero supports a free plan that individual contributors to OSS projects can utilize to make their contributions and/or to just test out the project.

Comments:
Thank you, but I don't think the project needs this right now. Appreciate the contribution :hugs:
Pixtral update example checkpoint
Author: amyeroberts
Pull request: yes

# What does this PR do?

Updates the checkpoint listed in the code example for Pixtral to one which won't error out 

Fixes #33490 #33591


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33633). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Support revision in trainer push_to_hub
Author: teamclouday
Pull request: no
Labels: name: trainer

name: Feature request

### Feature request

Would like to add a revision parameter in `push_to_hub` method of a trainer. The same parameter already exists in hub apis. https://github.com/huggingface/transformers/pull/25578

### Motivation

Would like to add tags or versions to the same model after training multiple times, so that it's easier to compare the performance. Adding revision tag will allow me to save multiple versions of same model in the same huggingface repo, instead of creating multiple repos.

### Your contribution

PR on the way

Comments:
[Backbone] Remove out_features everywhere
Author: NielsRogge
Pull request: yes

# What does this PR do?

As `out_features` is deprecated in favor of `out_indices` for the `xxxBackbone` classes, this PR removes a first batch of `out_features` mentions in favor of `out_indices`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33111). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Supoport batch image processing
Author: wcy1122
Pull request: no

### Feature request

A faster image processing which supports batch image processing, especially for input data like video.

### Motivation

The speed of image processing is too slow if the amount of image is large.

> 

### Your contribution

The image processor like CLIPImageProcessor processes each image sequentially, which makes it very slow.

Comments:
The input images are not necessary homogenous (unlike a tensor with batch dimension along which the elements are the same format).

You can definitely use `datasets` batch processing feature to speedup the  task.

cc @amyeroberts to see if they has any further suggestion/comment.

Hi @wcy1122, thanks for raising this feature request! 

As @ydshieh highlights, because the input images can be of different sizes and formats, and making them batchable may or may-not happen depending on the processor's configuration. Using `map` alongside `datasets` is a great way to parallize this process.

The image processors are, admittedly and unfortunately, very slow. Their principle purpose is to make it as easy as possible to go from an image to inputs which can be fed to the model i.e. a user can quickly test and get a prediction. 

Part of the reason for not engineering a highly-performant image processing library is that many great libraries already exist. You'll notice that in our training examples for vision models, [we use torchvision](https://github.com/huggingface/transformers/blob/3060899be51fe1a96b12de97376f2e2b8315bc4c/examples/flax/vision/run_image_classification.py#L338) for this very reason.

Get it. Thanks for the reply and suggestion. 

> Hi @wcy1122, thanks for raising this feature request!
> 
> As @ydshieh highlights, because the input images can be of different sizes and formats, and making them batchable may or may-not happen depending on the processor's configuration. Using `map` alongside `datasets` is a great way to parallize this process.
> 
> The image processors are, admittedly and unfortunately, very slow. Their principle purpose is to make it as easy as possible to go from an image to inputs which can be fed to the model i.e. a user can quickly test and get a prediction.
> 
> Part of the reason for not engineering a highly-performant image processing library is that many great libraries already exist. You'll notice that in our training examples for vision models, [we use torchvision](https://github.com/huggingface/transformers/blob/3060899be51fe1a96b12de97376f2e2b8315bc4c/examples/flax/vision/run_image_classification.py#L338) for this very reason.

yes， i use the dataset map with multiprocess to process CLIPImageProcessor, but when data process finished , the program hang, it seems that wait for something, long long time after, it is over because the time error , can you help to give a good example?
offload to CPU will increase memory
Author: zhuansmueoague
Pull request: no
Labels: name: bug

### System Info

latest

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

import torch
from transformers import AutoModel
import psutil
process = psutil.Process()

CPM_PATH  = "minicpm_v2"
minicpm_model = AutoModel.from_pretrained(CPM_PATH,
                                          trust_remote_code=True,
                                          torch_dtype=torch.bfloat16)
minicpm_model.to("cuda")

memory_info = process.memory_info()
memory_usage_MB = memory_info.rss / (1024 ** 2)
print(f"Memory usage: {memory_usage_MB}")


for i in range(10):
    minicpm_model.to("cpu")
    minicpm_model.to("cuda")

memory_info = process.memory_info()
memory_usage_MB = memory_info.rss / (1024 ** 2)
print(f"Memory usage: {memory_usage_MB}")


### Expected behavior

Memory usage: 720.6875
Memory usage: 6311.3515625


The memory should be same since the model is both on GPU at the beginning and the end. 

Comments:
Hi @zhuansmueoague, thanks for opening this issue! 

This is possibly a torch issue - calling `model.to(device)` just calls `nn.Module`'s  to method [under the hood](https://github.com/huggingface/transformers/blob/5bcbdff15922b1d0eeb035879630ca61c292122a/src/transformers/modeling_utils.py#L2884). To check if there's a memory leak - it'd be best to see how the memory used varies for each step in the for loop 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Out of memory when using phi3 for token classification
Author: xinyudong93
Pull request: no
Labels: name: bug

### System Info

I'm using AWS sagemaker to implement a token classification model using phi3

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

I got the OOM error message, when I try to import phi3-mini for token classification, even I use 1 sample per batch, it still exists.

    model =  Phi3ForTokenClassification.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    num_labels=9,
    id2label=id2label,
    label2id=label2id,
    use_cache=False,
    torch_dtype=torch.bfloat16
    )

    peft_config = LoraConfig(
        r=16,
        lora_alpha=16,
        target_modules=["up_proj",
        "gate_proj",
        "k_proj",
        "q_proj",
        "v_proj",
        "down_proj",
        "o_proj"],
        lora_dropout=0.1,
        bias="none",
        task_type=TaskType.TOKEN_CLS,
    )

    model = get_peft_model(model, peft_config)

tokenizer = AutoTokenizer.from_pretrained('microsoft/Phi-3-mini-4k-instruct',max_seq_length = 96)

    data_collator = DataCollatorForTokenClassification(tokenizer=tokenizer)
    
    output_dir = "/opt/ml/model/"
    
    training_args = TrainingArguments(
        output_dir=output_dir,
        bf16=args.bf16,
        learning_rate=2e-5,
        per_device_train_batch_size=1,
        num_train_epochs=1,
        weight_decay=0.01,
        report_to = "tensorboard",
        logging_dir=f"{output_dir}/logs",
        logging_strategy="steps",
        logging_steps=50,
    )

    trainer = Trainer(
        model,
        args=training_args,
        train_dataset=dataset,
        tokenizer=tokenizer,
        data_collator=data_collator,
    )

    trainer.train()




### Expected behavior

Can you help me to solve it or suggest what might be the problems.

Comments:
Hi, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

To get the best help I'd suggest sharing as much technical information as possible including all relevant error information e.g. full stack trace, observed memory utilization; and relevant technical information e.g. hardware being used. 

To help debug if it's related to your setup or the model, you can try running with a small model and see if it's successful. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Removing last element of class_queries_logits is not appropriate when do_reduce_labels is set to false.
Author: taghizad3h
Pull request: no
Labels: name: Vision

https://github.com/huggingface/transformers/blame/50837f20608e9266e3b9a930550fa6104230ccc7/src/transformers/models/mask2former/image_processing_mask2former.py#L998

I see that in the mentioned line we have
```
        masks_classes = class_queries_logits.softmax(dim=-1)[..., :-1]
```
which removes last class probability (may be the null class). It's not always true. For example if we don't set do_reduce_labels to true we need the background class as well. Or suppose we have two classes one for background and the other foreground. In that case when argmax is applied on dimension 1 afterwards the argmax always chooses 0 index because we have 1 element in that dimension. 

Comments:
Hi @taghizad3h, thanks for the issue!

Mask2Former [adds an additional class label](https://github.com/huggingface/transformers/blob/a5a8291ad10bda4933c1859b40155571567297f6/src/transformers/models/mask2former/modeling_mask2former.py#L2326) for the "void" class, it does not depend on the `do_reduce_labels` option and is needed to filter out queries without class (background can also be considered as a class). That is why we slice this "void" class from logits after applying softmax.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
when i update transformers from 4.38.1 to  version==4.42.0, it happened that Failed to import trl.models.modeling_base because of the following error (look up to see its traceback): '>' not supported between instances of 'NoneType' and 'str'
Author: ldh127
Pull request: no
Labels: name: bug

### System Info

when i update transformers from 4.38.1 to  version==4.42.0, it happened that Failed to import trl.models.modeling_base because of the following error (look up to see its traceback): '>' not supported between instances of 'NoneType' and 'str'

### Who can help?

@muellerzr @SunMarc

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

when i update transformers from 4.38.1 to  version==4.42.0, it happened that Failed to import trl.models.modeling_base because of the following error (look up to see its traceback): '>' not supported between instances of 'NoneType' and 'str'

### Expected behavior

@muellerzr @SunMarc

Comments:
Can you please give us the full trace? What import did you try?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Misleading type annotations in data collator
Author: sedol1339
Pull request: no

### Describe the bug

In `DataCollatorWithPadding`, the method `__call__` has the following sitnature:
```
def __call__(self, features: List[Dict[str, Any]]) -> Dict[str, Any]:
```
However, in the [official tutorial](https://huggingface.co/learn/nlp-course/en/chapter3/2) on line `batch = data_collator(samples)` the collator actually accepts not `List[Dict[str, Any]]`, but `Dict[str, List[Any]]`, that is, the already batched sample.

This complicates understanding collators and implementing custom ones.

### Steps to reproduce the bug

Run all steps in the tutorial:
https://huggingface.co/learn/nlp-course/en/chapter3/2

Check the `samples` variable that is passed to `data_collator`.

### Expected behavior

A more clear documentation and type annotation is desired.

### Environment info

datasets 2.20.0
Python 3.10.14
Ubuntu 22

Comments:
This is the `datastets` repository, and the `DataCollatorWithPadding` belongs to the `transformers` repository.

Hi @sedol1339, thanks for reporting! 

Would you like to open a PR fixing the typo? This way you get the github contrib for spotting & suggesting the fix 

I am not sure this is a typo. `DataCollatorWithPadding` do accept `List[Dict[str, Any]]`, as written in type annotation. But it also accept `Dict[str, List[Any]]` in some cases (see the example above).

@sedol1339 If the collator accepts both, then we can indicate this with a `Union` type annotation 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.