noamaanMulla-03/transformers-issues huggingface dataset:
Getting GPU crash when running compute_metrics within SFTTrainer
Author: matthewchung74
Pull request: no

### System Info

This is in colab https://colab.research.google.com/drive/1qTIxG_9R8xIYryY5mEPXIydZB5QeSnKp#scrollTo=5VM67iQMymv5


I have an Mistral Fine tune in Colab. Before training, I try running 

```
trainer.evaluate()
```
and get the following error message
```
[/usr/local/lib/python3.10/dist-packages/transformers/trainer_pt_utils.py](https://localhost:8080/#) in torch_pad_and_concatenate(tensor1, tensor2, padding_index)
     86 
     87     # Now let's fill the result tensor
---> 88     result = tensor1.new_full(new_shape, padding_index)
     89     result[: tensor1.shape[0], : tensor1.shape[1]] = tensor1
     90     result[tensor1.shape[0] :, : tensor2.shape[1]] = tensor2

OutOfMemoryError: CUDA out of memory. Tried to allocate 12.94 GiB (GPU 0; 39.56 GiB total capacity; 18.33 GiB already allocated; 12.29 GiB free; 25.71 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
```
however, when compute_metrics is commented out, i can evaluate/train the model

Any help is appreciated. Thanks!


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. run the colab https://colab.research.google.com/drive/1qTIxG_9R8xIYryY5mEPXIydZB5QeSnKp#scrollTo=5VM67iQMymv5

2. see the error
3. comment out comput_metrics
4. run again and do not see error

### Expected behavior

I don't see compute_metrics requiring any additional resources, so am unsure why it would be using more gpu and would expect it not to crash.

Comments:
cc @pacman100 @muellerzr

thank you @muellerzr 

I believe https://github.com/huggingface/transformers/pull/27458 will help with this after a bit more fixing in the PR

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Improve padding behaviour of `ClapFeatureExtractor`
Author: anmolojha
Pull request: yes

# What does this PR do?
Currently, `ClapFeatureExtractor`'s padding behaviour is unintuitive. Specifically, 
* It works with any value of argument `padding` without any check. 
* It uses non default padding strategy if `padding=True`.

This was discussed in more detail around [this](https://github.com/huggingface/transformers/issues/23648#issuecomment-1558586027) comment of issue #23648.

This PR closes #23648. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sanchit-gandhi 

Comments:
Hi @sanchit-gandhi! I need your advice on how can we create this PR without breaking anything. I have implemented what seems like the best way to me. Looking forward to hearing from you. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @anmolojha do you still want to work on this or should we close it? 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How to avoid the peak RAM memory usage of a model when I want to load to GPU
Author: JoanFM
Pull request: no

### System Info

- `transformers` version: 4.36.2
- Platform: Linux-5.10.201-191.748.amzn2.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.13
- Huggingface_hub version: 0.20.2
- Safetensors version: 0.4.1
- Accelerate version: 0.26.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

I am using transformers to load a model into GPU, and I observed that before moving the model to GPU there is a peak of RAM usage that later gets unused. I assume the model is loaded into CPU before moving into GPU.

In GPU model takes around 4Gi and to load it I need more than 7Gi of RAM which seems weird.

Is there a way to load it direcly to the GPU without spending so much RAM?

I have tried with the `low_cpu_mem_usage` and `device_map` parameter to `cuda` and `auto` but no luck.

```python
from transformers import AutoModel; m = AutoModel.from_pretrained("jinaai/jina-embeddings-v2-base-en", trust_remote_code=True, low_cpu_mem_usage=True, device_map="auto")
``` 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoModel; m = AutoModel.from_pretrained("jinaai/jina-embeddings-v2-base-en", trust_remote_code=True, low_cpu_mem_usage=True, device_map="auto")
```

### Expected behavior

Not having such a memory peak

Comments:
Hey 🤗  thanks for opening an issue! I am not sure you can prevent CPU usage (transfer from SSD to GPU), not sure anything supports it. However device_map = "auto" should always allow you to load the model without going over the ram usage. 
The peak can come from `torch. set_default_dtype(torch.float16)` and the fact that you are not specifying a dtype. So the model might be loaded in float32, then casted then transfered. 

so what would you actually suggest  to do? What `dtype` parameter should I pass?

`float16` or something like that. Or use TEI https://huggingface.co/docs/text-embeddings-inference/index  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Thanks for helping, it was indeed an issue with dtype
[Nougat] Fix pipeline
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR ensures [Nougat](https://huggingface.co/docs/transformers/main/model_doc/nougat) works with the image-to-text pipeline.

Fixes #27475

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28242). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Docs] Update README and default pipelines
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR makes sure the main README is a bit more up-to-date, and updates the default recommendations for depth estimation and zero-shot image classification/object detection.

Fixes #28762

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28864). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update the default depth estimation model of the pipeline
Author: NielsRogge
Pull request: no

### Feature request

The current depth estimation pipeline leverages https://huggingface.co/Intel/dpt-large by default.

However, with recent models like Depth Anything, it might make sense to update the default model.

For reference, DPT-large (also called DPT 3.0) was released in 2021, we do have better models in 2024 :)

### Motivation

Having better depth estimation models by default would be great.

### Your contribution

I could contribute this, but let's perhaps first discuss

Comments:
cc @Narsil @amyeroberts I would propose to update the default depth estimation model to https://huggingface.co/LiheYoung/depth-anything-base-hf. Any objections?



In principle, I think this might be a good idea: people want to just get good predictions, and serving the "best" model by default is probably improving user experience. However, it does change things silently for the users. 

There's a few questions this brings up: 
* Backwards compatibility - users who were calling the default pipeline will see their predictions change and possibly compatibility with the environment and hardware. If things suddenly stop working then it's a bad user experience.  
* What's the reason for not upgrading to different models for the other pipelines? I can see they're all using older / more basic models like vit and distilbert. Simplicity and size might be a reason for having a default model: a pipeline call anyone can run is better than one that needs e.g. a large machine. 
* What are the rules for updating models? Is anyone responsible? What are the criterea for a new model to become the default? 

Afaik we don't have rules around changing default models.

So far, I think the main thing has been routing users to call pipeline on specific models rather than task themselves (so they are responsible for upgrading their model, and we are not so we don't have to worry about backward compatibility).

I think the best course of action is to promote 

```python
pipeline(model="LiheYoung/depth-anything-base-hf")
```

instead of 

```python
pipeline("depth-estimation")
```

Updating the docs (So the default model within the docs) would be non breaking and still be up to date for new users.

cc @Rocketknight1
### Summary
Author: goalend
Pull request: no

### Summary

The importance of enforcing a set of quality standards to continuously deploy in a consistent and predictable way can’t be underestimated. Implementing these standards without the duplication of CI/CD configuration code is a challenge many organizations face today. How can workflows solve these problems and allow the ability to “push things down” or enforce requirements from organization down to its repositories?

We are building the controls that allow an organization to require a workflow file (or list of workflow files) to pass before code is merged into any of its repositories. Requiring a workflow to pass before merging will be available via Rulesets and will allow access to controls already available to branch rulesets, such as:
- enforcement status
- bypass rules
- repository targeting

**By requiring a workflow to pass before merging will allow organization admins to specify:**
- which workflow file to run from any of the organization’s repositories
- a specific branch or tag for the workflow file
- an exact commit sha to pin to the required workflow _(optional)_


**Required workflows can help companies with the following use cases:**
- Correctness and Compliance: Ensure that all code meets an enterprise’s quality standards before merging.
- DRY: Reducing duplication of CI/CD configuration code


- [ ] - _Originally posted by @github-product-roadmap in https://github.com/github/roadmap/issues/638__[@]()_

Comments:
✌️
Make it possible to  pass `torch_dtype` argument as a string so it's easier to serialize
Author: tokestermw
Pull request: no
Labels: name: Feature request

### Feature request

Hi, it would be nice if we can pass `torch_dtype` as a string.

```python
from transformers import pipeline

# this
m = pipeline('text-generation', 'Mistral-7B-Instruct-v0.1', device=0, low_cpu_mem_usage=True, torch_dtype=torch.float16)

# or that
m = pipeline('text-generation', 'Mistral-7B-Instruct-v0.1', device=0, low_cpu_mem_usage=True, torch_dtype='float16')
```

### Motivation

* It's common to use a config file (e.g. yaml) to pass in arguments to `from_pretrained`, but `torch_dtype` requires `torch`. e.g. we can't use the `assemble` method in [spacy-llm](https://github.com/explosion/spacy-llm/blob/7a0460ce112ae1fe783dd100b4d32e56c282919f/spacy_llm/util.py#L37).
* For small GPUs (e.g. T4), we can't load any models 7B and higher (cuda memory error) unless specifying `torch_dtype=torch.float16` (`torch_dtype='auto'` doesn't work either) (assuming not using 8-bit quantization or lower).


### Your contribution

Yes can make a PR so if `torch_dtype` is a string, we convert to `torch` type.

e.g. `'float16'` to `torch.float16`


Comments:
I am happy to take a look at this!
Allow setting dtype in rescaling in image_processing_donut.py
Author: archit76
Pull request: yes

Fixes #28969 
With this fix, dtype can be changed by passing as a kwargs argument

if do_rescale:
    images = [
        self.rescale(image=image, scale=rescale_factor, input_data_format=input_data_format, **kwargs)
        for image in images
    ]

Comments:
Not required anymore. Works for me!
Updated requirements for image-classification samples: datasets>=2.14.0
Author: alekseyfa
Pull request: yes

# What does this PR do?

This PR updates the dependency requirements for image-classification case.

The run_image_classification.py script in the current implementation uses the token parameter, which was introduced as of datasets version 2.14.0. Thus, using packages below the specified version may cause errors.

Please see the release link below:
https://github.com/huggingface/datasets/releases/tag/2.14.0

An example of using the argument token in the run_image_classification.py script is shown below:

```
  if data_args.dataset_name is not None:
      dataset = load_dataset(
          data_args.dataset_name,
          data_args.dataset_config_name,
          cache_dir=model_args.cache_dir,
          token=model_args.token,
      )
```

https://github.com/huggingface/transformers/blob/main/examples/pytorch/image-classification/run_image_classification.py#L260


A similar PR has already been merged into the optimum-habana repository:
https://github.com/huggingface/optimum-habana/pull/699


<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28974). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Tests: tag `test_save_load_fast_init_from_base` as flaky
Author: gante
Pull request: yes

# What does this PR do?

As discussed [internally on slack](https://huggingface.slack.com/archives/C01NE71C4F7/p1707407250079089) with @ydshieh -- this test is known to be flaky for a while.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28930). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

>  no matter what the text is put I will accept :-)

@ydshieh 

```
@is_flaky(description="is flaky")
```

;) 

FYI:

> Hi! The following are the extra failures your branch rebased on ae0c27ad against CI on ae0c27ad (main).

> > no matter what the text is put I will accept :-)
> 
> @ydshieh
> 
> ```
> @is_flaky(description="is flaky")
> ```
> 
> ;)

You are a debug master now @amyeroberts !
[Docs] Add video section
Author: NielsRogge
Pull request: yes

# What does this PR do?

Video models deserve their own section in the docs, as they're currently hidden among the dozens of vision models.

cc @stevhliu 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28958). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Docs] Add language identifiers to fenced code blocks
Author: khipp
Pull request: yes

# What does this PR do?

This PR adds missing language identifiers to fenced code blocks to ensure consistent syntax highlighting in the generated documentation.

For example, see the differences between the first and last block when the identifier is missing:
<img width="908" alt="image" src="https://github.com/huggingface/transformers/assets/9824526/6cf5cea5-3e6b-4a50-8d5a-7da7c61960c4">

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Documentation: @stevhliu


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28955). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Explicitly check if token ID's are None in TFBertTokenizer constructor
Author: skumar951
Pull request: yes

# What does this PR do?

Since the token ID's are ints, the `if token_id` check will fail if `token_id == 0`, when it should only fail when `token_id == None`. This PR adds an explicit None check.

@ArthurZucker


Comments:
Failing test is unrelated I'll merge
[`NllbTokenizer`] refactor with added tokens decoder
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes #26497 by make the list of languages optional. By default these languages will be added, but otherwise by setting `additional_special_tokens = None` can be suppressed in both the fast and slow tokenizer. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27717). All of your documentation changes will be reflected on that endpoint.

@amyeroberts failing test is flaky, no idea why it's behaving that way will check main in the mean time

Regarding failing tests - there's an internal thread here: https://huggingface.slack.com/archives/C01NE71C4F7/p1705082667041509 cc @ydshieh 

`=========================================================================== 97 passed, 4 skipped, 9 warnings in 75.13s (0:01:15) ===========================================================================`
slow tests all pass ready to merge! 

This breaks the initialization as you will not always have the extra langages. But it's important to use the new `added_tokens_decoder` behaviour for that!
[i18n-fr] Translate pipeline tutorial to French
Author: NoB0
Pull request: yes

# What does this PR do?

Translates the `pipeline_tutorial.md` file of the documentation to French.

Part of #21456

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

French speaking contributors.
Documentation: @stevhliu and @MKhalusova

Comments:
> Pipeline is a female gendered noun in french! Otherwise should be alright

According to [Larousse](https://www.larousse.fr/dictionnaires/francais/pipeline/61067) and [Le Petit Robert](https://dictionnaire.lerobert.com/definition/pipeline) dictionaries it is male gendered. Waiting for feedback before changing the gender.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28359). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

It is not the same object 😅 
> anglicisme Tuyau servant au transport à grande distance et en grande quantité de fluides (pétrole, gaz naturel…). ➙ [gazoduc](https://dictionnaire.lerobert.com/definition/gazoduc), [oléoduc](https://dictionnaire.lerobert.com/definition/oleoduc).

this one is male, but it sounds wrong when talking about AI no? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.