noamaanMulla-03/transformers-issues huggingface dataset:
Add test for parse_json_file and change typing to os.PathLike
Author: xu-song
Pull request: yes

# What does this PR do?


Fixes # (issue)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Question about the use of bias in the Graphormer Model
Author: sarah-af
Pull request: no

Hi,

The documentation of the Graphormerconfig indicates that the parameter,
bias (bool, optional, defaults to True) — Uses bias in the attention module - unsupported at the moment.

I have 2 questions,
1. Is that the same attention bias introduced in the paper using the shortest path distance? Or where is it applied?
2. What does unsupported mean? I see in the src files that the attention bias has been implemented.

Many thanks!

Comments:
cc @clefourrier 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @sarah-af ,
1. The bias is applied in the MultiHeadAttention ([here](https://github.com/alexanderkrauck/transformers/blob/abaca9f9432a84cfaa95531de4c72334f38a42f2/src/transformers/models/graphormer/modeling_graphormer.py#L328) for example), which is used for self attention computations. From browsing the paper again (it's been a long time :sweat_smile: ) I think it is indeed the attention bias you mention.
2. I had initially not added bias management in the Graphormer v0, and forgot to update the docs when adding it. If you have the time, feel free to open a PR to fix this, it would be very useful :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix `filter_models`
Author: ydshieh
Pull request: yes

Fix #29683


# What does this PR do?

#29682 is reverted because, when there are many files impacted and `filter_models=False`, the argument to pytest is too long, and `fetch_tests` job on Circle fails with `/bin/bash: line 25: /usr/bin/jq: Argument list too long`.

This PR avoids this situation by using a heuristic of 30 models. If there are more than 30 models being impacted +`filter_models=False`, we will trigger a full CI by using `tests` as the target.

See the file `tests_fetched_summary.txt` in

https://app.circleci.com/pipelines/github/huggingface/transformers/87349/workflows/3a84543d-d965-416d-81f7-eee0286e30c7/jobs/1134895/artifacts

```bash
More than 30 models are impacted and `filter_models=False`. CI is configured to test everything.

### TEST TO RUN ###
- tests
```



Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29710). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Don't use a subset in test fetcher if on main branch
Author: ydshieh
Pull request: no

### Feature request

Don't select a subset from the detected tests to run when they are many - if we are on the main branch.
This could detect any issue as early as at the merge time, not at the nightly run.

This is implemented in #28816 / #29673, but reverted in #29682 as we have 

> /bin/bash: line 25: /usr/bin/jq: Argument list too long
Exited with code exit status 126

as there are too many models and the arguments to pytest is too long.

We should find a way to overcome this in order to achieve our goal.

Comments:
Fixed in #29710
More fixes for doctest
Author: ydshieh
Pull request: yes

# What does this PR do?

I am just trying to fix more that I could for doctests.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30265). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
FIX: Fix 8-bit serialization tests
Author: younesbelkada
Pull request: yes

# What does this PR do?

with the recent safetensors refactor passing pure str in state dicts are no longer supported. This PR together with https://github.com/TimDettmers/bitsandbytes/pull/1164 should fix the issues

cc @ArthurZucker @Narsil @SunMarc @Titus-von-Koeller 

Comments:
> with the recent safetensors refactor passing pure str in state dicts are no longer supported

It was never supported. The strings where silently dropped. Now it's a strong error (which I think is better behavior since the safetensors version was loosing that information which would have lead to incorrect reloads)

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30051). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks! 
I had a deeper look actually, it seems `weight_format` was never used in our case since before the ST PR that value from the state dict was popped and ignored. The correct weight format is inferred after the first forward pass
Now that we do have a hard check we simply need to make sure we save uint8 tensors instead of pure str: https://github.com/TimDettmers/bitsandbytes/pull/1164 + make sure to ignore them when loading the state dict as done in this PR
TextStreamer: An option to print / put every single token instead of whole words
Author: drummerv
Pull request: no
Labels: name: Feature request

name: Generation

### Feature request

https://github.com/huggingface/transformers/blob/v4.37.2/src/transformers/generation/streamers.py#L108-L114

The current implementation forces us to visualize decoding on a per word basis. Add an option to print the string as soon as it's generated.

### Motivation

I use TextStreamer to visualize several things in my terminal:
1. How words are split
2. Actual token-per-second performance
3. Understand how it generates very long words like URLs

### Your contribution

I'm a Python noob

Comments:
cc @gante 

@vicboyv it's in our plans, but I can't give you a timeframe :) 

I will also find it very helpful. Also I have a related question. If I pass to the model my own streamer - and modify the "put()" function accordingly, don't I solve the problem simply by recording the time each time?
Thanks!
`TextIteratorStreamer` & `generate` graceful interruption
Author: paulcjh
Pull request: yes
Labels: name: Generation

# What does this PR do?

This PR adds in functionality to stop the `generate` function for LLMs when used in conjunction with a streamer. The `TextIteratorStreamer` typically runs in a separate thread , per the example provided in its docstring, and if you need to terminate the generation there's currently no way to do this. This makes operating the models in an API environment challenging when things such as client disconnects occur.

_note: the file was also formatted with black, happy to remove and just keep the new interruption changes_

## Before submitting
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@gante 


Comments:
FYI: it looks like you're including a lot of changes to generation/utils.py that are just formatting changes. I'm guessing this sort of thing is discouraged since it makes it difficult to identify the substantive changes you are proposing, and also is likely to cause merge conflicts for others. Concretely: I was curious to see what changes you had made in utils.py, but between the formatting changes and collapsing all of your work into a single commit, your substantive changes in that file are a needle in a haystack.

@dmarx just removed all of the cleanup - the changes in `utils.py` are very light

@gante do you have any thoughts on this feature?

Hi @paulcjh 👋 Thank you for opening the PR!

We're planning to rework streaming soon, so I'm avoiding adding short-lived changes. Have a look at the conversation with @dmarx on [this PR](https://github.com/huggingface/transformers/pull/29545), I think it covers the reasons to not add changes and the roadmap for streaming 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Allow for str versions of dicts based on typing
Author: muellerzr
Pull request: yes

# What does this PR do?

This PR adds support for passing in a string dictionary to arguments that allow a dict in the argparser. For example:

```
python test.py --output_dir testdir --accelerator_config='{"dispatch_batches":"False"}'
```
(`test.py` is just a small script reading the args from the parsers):
```python
from transformers import HfArgumentParser, TrainingArguments

parser = HfArgumentParser((TrainingArguments))

training_args = parser.parse_args_into_dataclasses()

print(training_args)
```

This also fixes issues with typing of not being able to state that `deepspeed` and `fsdp_config` can't have type `dict`. Turns out it's a (very) fun setting with the argparser wrapper we have for these. Types should be declared as (for this):
```python
Optional[Union[dict,str,...]]
```
This is **very** important to get working properly!


Fixes https://github.com/huggingface/transformers/issues/30204


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@amyeroberts @pacman100 

Technically everything will go brr if passes, since any call to the CLI/parser happens usually with examples etc. So new tests added, but did manually verify input/output from a basic script locally

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30227). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts figured out where to add a test, and verified that all 3 conditions raise their respective errors :)
Enhance HfArgumentParser with Dict command-line parser
Author: xu-song
Pull request: no
Labels: name: trainer

name: Feature request

### Feature request

Enhance HfArgumentParser with Dict command-line parser

### Motivation

There are many arguments with dict typing, such as `lr_scheduler_kwargs`, `gradient_checkpointing_kwargs`.

https://github.com/huggingface/transformers/blob/6cdbd73e01a9719bfaec07d91fd108e8d932bbbb/src/transformers/training_args.py#L806

https://github.com/huggingface/transformers/blob/6cdbd73e01a9719bfaec07d91fd108e8d932bbbb/src/transformers/training_args.py#L1255

It is possible to pass dict arguments through json_file or yaml_file.
But it does not work in command-line.

### Your contribution

TBD

Comments:
cc @pacman100 @muellerzr
Workflow: Update tailscale to release version
Author: younesbelkada
Pull request: yes

# What does this PR do?

For the new push important models workflow, we were using a dev version of the tailscale GHA, now that a proper v1 has been released, we should switch to it - as discussed offline with @glegendre01 

cc @glegendre01 @ydshieh @amyeroberts 


Comments:
No question about this PR itself. But I have a related question: why if we want to ssh to the runner even before the pytest is running? Isn't it better to put the ssh job at the beginning (if we are in debugger mode)?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30268). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

good point @ydshieh 

> No question about this PR itself. But I have a related question: why if we want to ssh to the runner even before the pytest is running? Isn't it better to put the ssh job at the beginning (if we are in debugger mode)?

not sure this is possible, @glegendre01 might know best here 🙏
add `push_to_hub( )` method when working with pipelines
Author: not-lain
Pull request: no
Labels: name: Feature request

### Feature request

it will be good to addition to the library to add a `push_to_hub( )` method when working with the pipeline method.

### Motivation

I have just finished up a blogpost about creating custom architectures, but i couldn't help but notice that when working with ` AutoConfig` or `AutoModelForxxx` or `PreTrainedModel`  have a `push_to_hub( )` methods alllowing them to easily be updated on the hub.
unfortunately, this isn't the case for the pipeline method, I found a possible work around by cloning the repo then using `save_pretrained( )` then pushing the changes, but this not optimised and takes lots of time especially when working with really large language models
check the documentation : 
https://huggingface.co/blog/not-lain/custom-architectures-with-huggingface#push-to-hub-%F0%9F%A4%97
<hr>

![image](https://github.com/huggingface/transformers/assets/70411813/11a3f62f-0a39-48b1-9109-0e7c3896f526)
**VS**
![image](https://github.com/huggingface/transformers/assets/70411813/09562086-a0f5-4bb9-86c9-57b92ec637b1)


### Your contribution

yes, I can help out with a PR

Comments:
FYI @Rocketknight1
Raise relevent err when wrong type is passed in as the accelerator_config
Author: muellerzr
Pull request: yes

# What does this PR do?

Currently we don't raise an error if a user passes in something like `AcceleratorConfig` to the `accelerator_config` arg instead of `AcceleratorConfig()` (so they pass in the type, **not** an object).

This will raise an error if so, catching it early for downstream libs. 

Fixes https://github.com/huggingface/transformers/issues/29993


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29997). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
BLIP - fix pt-tf equivalence test
Author: amyeroberts
Pull request: yes

# What does this PR do?

Fixes failing test on main after merge of #30253 

Failing run: https://app.circleci.com/pipelines/github/huggingface/transformers/89639/workflows/5b9e1145-97cd-452c-b36f-be6780d11f87/jobs/1168209

`RUN_PT_TF_CROSS_TESTS=1 pytest tests/models/blip/test_modeling_blip.py::BlipTextImageModelTest::test_pt_tf_model_equivalence ` run and confirmed to be passing with this change 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30258). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Merging as this resolves tests failing on `main`
Fix test fetcher (doctest) + `Idefics2`'s doc example
Author: ydshieh
Pull request: yes

# What does this PR do?

- test fetcher (doctest): a mistake I made in the previous PR
- `Idefics2`'s doc example: not able to fit into GPU even with fp16. So I changed it.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30274). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix tests for Vision LLMs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

 IDEFICS-2 pull request has added MODEL_FOR_VISION_2_SEQ_MAPPING_NAMES to the preparation of inputs in "test_modeling_common.py". That started causing test failures for composite vision language models, which do not have "seq_length" in their tester. This PR adds "seq_length" in a similar way "batch_size" was added to these models before.

All tests for the changed models are passing locally. Llava and other VLMs do not have a composite structure, so the tests are passing without a fix.
 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30270). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for opening @zucchini-nlp and addressing so quickly! This should be fixed on main. I had a PR open to addressed - #30258 - and was waiting for review. Decided to merge to avoid red on main

Ah I see, thanks. I'll close this PR then :)
Fix SDPA sliding window compatibility
Author: fxmarty
Pull request: yes

As per title, fixes https://github.com/huggingface/transformers/issues/28980

Supersedes https://github.com/huggingface/transformers/pull/29220 https://github.com/huggingface/transformers/pull/29407 as the implementation ends up being different (added you as co-author here @ehuaa).

This bug dates back to https://github.com/huggingface/transformers/pull/26572 where `sliding_window` was not properly accounted for in the `_prepare_4d_causal_attention_mask_for_sdpa` method. Since then, SDPA support was added to models that use sliding window, but this bug was not yet fixed.

Comments:
Hi, @fxmarty ,thanks for your great work! I think you can add some tests i mentioned in https://github.com/huggingface/transformers/pull/29407, to check if the result with sliding window in SDPA is the same as flashattention2.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30127). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ehuaa Thank you, for sure will do!

Runing mistral, mixtral, starcoder2 tests, those fail but are already failing on main:
```
FAILED tests/models/mistral/test_modeling_mistral.py::MistralIntegrationTest::test_speculative_generation - AssertionError: 'My f[19 chars]is 100% Sriracha. I love the heat, the tang and the fact costs' != 'My f[19 chars]is 100% Sriracha. I love the heat, the ...
FAILED tests/models/mixtral/test_modeling_mixtral.py::MixtralIntegrationTest::test_small_model_logits - AssertionError: The values for attribute 'dtype' do not match: torch.float16 != torch.float32.
FAILED tests/models/mixtral/test_modeling_mixtral.py::MixtralIntegrationTest::test_small_model_logits_batched - AssertionError: The values for attribute 'device' do not match: cuda:0 != cpu.
FAILED tests/models/starcoder2/test_modeling_starcoder2.py::Starcoder2IntegrationTest::test_starcoder2_batched_generation_4bit - AssertionError: Lists differ: ['Hel[110 chars]t is related to the topic of "How to make a ga[179 chars]ute'] != ['Hel[110 chars]t is aimed at creating a...
FAILED tests/models/starcoder2/test_modeling_starcoder2.py::Starcoder2IntegrationTest::test_starcoder2_batched_generation_eager - AssertionError: Lists differ: ['Hel[181 chars]I am currently working on', "def hello_world()[114 chars]app"] != ['Hel[181 chars]I am looking for a', "de...
```