noamaanMulla-03/transformers-issues huggingface dataset:
Revert workaround for TF safetensors loading
Author: Rocketknight1
Pull request: yes

In #30118 @amyeroberts skipped safetensors loading on TF models. I believe I've worked out the cause, and it was that the safetensors weights in that repo were a bit malformed. I tried fixing them by loading and saving them again with a torch `AutoModelForCausalLM`, so hopefully we can use the fixed weights and revert the changes.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30128). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

After testing on the PR branch, this looked good, so I merged the changes to `hf-internal-testing/tiny-random-gpt2`

All the actual tests are passing, but we have some rate limit errors on other tests. Raised the issue internally!

Merging as the failing tests are unrelated
[Trainer] Fix default data collator
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR isolates a fix proposed in #30129, namely the DataCollatorWithPadding only needs to be used in case a tokenizer is being passed and is effectively a tokenizer or feature extractor, not another object like an image processor.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30142). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Trainer] Undo #29896
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR reverts #29896 as a better way to handle it is #30102, but that one requires a huge amount of code updates + more extensive offline discussion as it's quite a big change to something people use a lot.

Therefore, a better fix for #29790 is simply this:

```
default_collator = (
            DataCollatorWithPadding(tokenizer)
            if tokenizer is not None and isinstance(tokenizer, PreTrainedTokenizerBase)
            else default_data_collator
        )
```


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30129). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Ok opened a separate PR for this
Send headers when converting safetensors
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix some torch pipeline failing due to the too many requests to the Hub because the correct header is not used.

Great @Wauplin fixed everyting, I am just opening the PR here.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30144). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix quantization tests
Author: SunMarc
Pull request: yes

# What does this PR do ? 

This PR fixes all failing tests in the quantization CI. We also split the workflow so that we have one job per quantization method and device. 
You can see that everything works fine from
- the run page: https://github.com/huggingface/transformers/actions/runs/8483098195/job/23243697001
- the report (internal): https://huggingface.slack.com/archives/C06KVUMU5JA/p1711728524491529


cc @ydshieh @younesbelkada 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29914). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thank you @SunMarc on this work, super appreciated!

From the run page: https://github.com/huggingface/transformers/actions/runs/8483098195/job/23243697001
and the report (internal): https://huggingface.slack.com/archives/C06KVUMU5JA/p1711728524491529
(it would be nice if those are provided in the PR description),

I don't have specific question but just a slight question on the notification stuff.

Hi @SunMarc 

If you are OK, I would merge #30012 today. I can help to make the necessary changes to this PR to resolve the conflicts (or make it compatible to #30012).

WDYT?

Feel free to merge your PR @ydshieh ! Yes, that would be great if you could resolve the conflits if it is not too complicated for you ! 

Happy to help - if it is difficult for me , I guess it would be even difficult to you 😆 (to understand what I changed in my PR)

@ArthurZucker No need to review until I request a reivew  (after  rebasing this PR on #30012)

Back to this next week.
[docs] Fix image segmentation guide
Author: stevhliu
Pull request: yes

Fixes #30058:

- adds import for Accelerate
- adds code snippet to view image but also keeps viewing the metadata as well since imo its important to see the other fields in there
- reloads the dataset in the Inference section
- adds the `ade_palette` colormap manually so the code snippet just runs (let me know if there is a better way to import the colormap!)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30132). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[CI] Fix setup
Author: SunMarc
Pull request: yes

# What does this PR do ?

This PR skip the setup steps for some job since they don't need it. This regression was introduced in this [PR](https://github.com/huggingface/transformers/pull/29914) cc @ydshieh .
The condition works fine : https://github.com/huggingface/transformers/actions/runs/8618527519

Comments:
I'm going to merge this PR. @ydshieh said that it was okay since we have tiny changes. 

he is so fast anyway 😆
feat: Upgrade Weights & Biases callback
Author: parambharat
Pull request: yes

# What does this PR do?
This PR adds a few new functionalities to the Weights & Biases Callback

- Logs Peft and Lora Config to wandb if present
- Adds model parameter counts to wandb config and artifact metadata
- Adds on_predict methods to log prediction metrics
- Prints the model architecture to a file alongside the wandb artifact
- Logs initial and final models to the wandb artifact to full reproducibility
- Adds steps and epoch aliases to checkpoint artifacts

- Here's a [link](https://wandb.ai/parambharat/test-transformers/artifacts/model/model-rg4pcjcv/v3) to the what the logged artifacts look like
- Here's a run [overview page](https://wandb.ai/parambharat/test-transformers/runs/rg4pcjcv/overview?workspace=user-parambharat) with added config and metadata for the run with peft configs logged


## Before submitting
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?

## Who can review?

- trainer: @muellerzr and @pacman100

Comments:
@muellerzr and @ArthurZucker : Thank you for the review and comments. I have responded to the review comments with some explanations. Let me know if it looks good. 

@parambharat can you resolve the conflict? Thanks!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29125). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@muellerzr : Resolved :+1: 

cc @amyeroberts for a final review as she's on watch this week 🤗 

@amyeroberts . Thanks for the detailed comments and review. I have addressed all the comments and responded to them. Please take a look and let me known if there is anything pending. 

Hi @amyeroberts , Just checking in on this PR. Let me know if you have any comments and if my responses to the review comments are accepted and resolve them.

@parambharat Can you rebase on main to include any upstream changes? 

> @parambharat Can you rebase on main to include any upstream changes?

@ammerged: I main into the branch and pushed the changes.

@parambharat Could you try again rebasing? We can't merge until all the tests are 🟢  At the moment, these don't seem related to this PR, but I haven't observed them failing on other open PRs

@amyeroberts, @muellerzr  : It looks like rebasing with main pulled a bunch of commits and changes into the branch. This was unintentional. I'm closing this PR and raising a new one with just the changes relevant to the integration.

@amyeroberts and @muellerzr : Please find the new PR [here](https://github.com/huggingface/transformers/pull/30135)

@parambharat For future PRs, when many commits are added like this, it normally indicates that the force push wasn't used when pushing the rebased branch to the remote. As rebasing is effectively rewriting the history, it's necessary to do `git push -f` to have the rebase changes reflected.
Add datasets.Dataset to Trainer's train_dataset and eval_dataset type hints
Author: ringohoffman
Pull request: yes

# What does this PR do?

Adds `datasets.Dataset` to `Trainer`'s type hints for `train_dataset` and `eval_dataset`.

It is supported and mentioned by name in the docstrings for these parameters, but is not hinted as an acceptable type. Currently, if you try to call `Trainer` with a `datasets.Dataset` for one of these parameters, type hinters like pyright raise an error.

<!-- Remove if not applicable -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->

@amyeroberts
@muellerzr
@pacman100


Comments:
Rebasing onto main should resolve the unrelated test errors
Fix length related warnings in speculative decoding
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

Currently if we pass a `min_length` or `min_new_tokens` to speculative decoding, we get a bunch of warnings that 
`UserWarning: Unfeasible length constraints: min_new_tokens (34), when added to the prompt length (66), is larger than the maximum possible length (75)....`

This PR adds a `min_new_tokens` argument in candidate's generate, which will default to 0 if no `min_length` as passed by the user. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante 

fixes #29860


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29585). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp It looks like a lot of tests are failing at the moment because of the lack of `min_length` attribute 

Oops, I accepted the last suggestion and did not fix naming in other places of the code. Now it should work, at least locally it was passing tests in "generation"

@amyeroberts this one is ready to re-review 😃 

@amyeroberts can you merge pls? Failing tests if TF seem to be unrelated

As the failing tests are for generation, there's a very small chance there would be some interaction between the changes here and those tests (mainly because the test implementation isn't TF specific). 

They should now be resolved on `main` and resolve with a quick rebase :)
`assisted_decoding` called directly inside `generate` triggering warning to use when it shouldn't
Author: ofirzaf
Pull request: no
Labels: name: Generation

### System Info

- `transformers` version: 4.39.1
- Python version: 3.10.14
- Huggingface_hub version: 0.22.0
- Safetensors version: 0.4.2
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.1+cpu (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@gante 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
out = model.generate(
    **inputs,
    max_new_tokens=128,
    streamer=TextStreamer(tokenizer=tokenizer, skip_special_tokens=True),
    pad_token_id=tokenizer.eos_token_id,
    prompt_lookup_num_tokens=3,
)
```
This triggers the following warning:
```
Calling `_assisted_decoding` directly is deprecated and will be removed in v4.41. Use `generate` or a custom generation loop instead.
```
This happens probably due to the following call which should be changed to `self._assisted_decoding(...)`
https://github.com/huggingface/transformers/blob/7eb3ba82241c927053689270a0751f4ff5d33c54/src/transformers/generation/utils.py#L1509

### Expected behavior

Warning shouldn't be triggered

Comments:
Thanks for opening an issue. Yes, there is [a PR](https://github.com/huggingface/transformers/pull/29585) fixing this behavior. You can update `transformers` with 
`!pip install --upgrade git+https://github.com/huggingface/transformers.git`

after the PR is merged 🤗
[Minor] Fix type annotations for Cache
Author: huyiwen
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


Fix incorrect type annotations and incomplete type branching.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@gante, @stevhliu and @MKhalusova


Comments:
Hey @huyiwen 👋 Thank you for opening this PR! 

We are still working on the cache refactor, and a few things are still planned to be changed. These changes will render this PR obsolete and, as such, I won't be merging it. I hope you understand 🤗 

Hi, @gante . Where can I preview these refactors? I'm developing based on cache.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Cannot run flash_attention_2 with bark-small
Author: damithsenanayake
Pull request: no

### System Info

I'm trying to run bark / bark-small with flash-attention 2. It simply gives me the following error. I'm using the dev version. 
TypeError: BarkModel._check_and_enable_flash_attn_2() got an unexpected keyword argument 'check_device_map'. Was this supported in an earlier version? 

### Who can help?

@sanchit-gandhi 

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Simply follow the steps mentioned in https://huggingface.co/docs/transformers/en/model_doc/bark#using-flash-attention-2. 

### Expected behavior

Bark Model should be set up without an issue. 

Comments:
could you additionally share the output of `transformers-cli env` ? cc @ylacombe as well

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Invalid Version Error with Packaging Library for Torch Version "2.0.0.nv23.05"
Author: Enes-Yaman
Pull request: no

We are encountering an issue where the packaging library is unable to parse the Torch version string "2.0.0.nv23.05". This problem arises when trying to import the transformers library, specifically during the dependency version check for Torch. The error suggests that the version string does not conform to the expected format.

### System Info

Environment:

Device: NVIDIA Jetson AGX Orin
Docker Image: dustynv/ros:noetic-pytorch-l4t-r35.3.1
Python Version: 3.8.10
CUDA : 11.4.239
cuDNN : 8.4.1.50
Jetpack: 5.0.2 GA
Torch Version: 2.0.0.nv23.05

### Who can help?

_No response_

### Reproduction

1. Use the NVIDIA Orin device with the specified Docker image (dustynv/ros:noetic-pytorch-l4t-r35.3.1).
2. Attempt to import the transformers library in a Python 3.8 environment.
3. The import process fails with the following traceback:
`Traceback (most recent call last):
  File "/src/main.py", line 17, in <module>
    from segmentor import Segmentor
  ...
  File "/usr/local/lib/python3.8/dist-packages/packaging/version.py", line 198, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '2.0.0.nv23.05'
`

### Expected behavior

The packaging library should be able to parse the Torch version string "2.0.0.nv23.05" without raising an InvalidVersion error, allowing the transformers library to be imported successfully.

Comments:
Hey! I don't think this is related to transformers specifically 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.