noamaanMulla-03/transformers-issues huggingface dataset:
[WIP] Add H3
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds the H3 model by Hazy Research (Stanford University).

I've removed the Flash Attention dependency, and main author @DanFu09 has removed the einops dependency (🙏 ).

I've kept an optional soft dependency on `pykeops`, to allow for speedups. The model runs fine if the user doesn't have this library installed.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22262). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @NielsRogge ✋,

Nice work! Do you know if this model would be integrated in the near future inside HuggingFace? Was this PR staled given complexities with custom ops?

Could you give an overview of the missing steps needed in this PR to have a functional H3 model integrated into HF? 🙏 
Thanks for your work! 🙌 


Hi @gaceladri the PR is actually totally ready, the only thing that needs to done is perhaps make [this function](https://github.com/NielsRogge/transformers/blob/5199d3d3a08264f1b17442504559c28304ce619c/src/transformers/models/h3/modeling_h3.py#L139) more like the other Attention classes in the library (like [this class](https://github.com/huggingface/transformers/blob/17a55534f5e5df10ac4804d4270bf6b8cc24998d/src/transformers/models/llama/modeling_llama.py#L158)).

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @NielsRogge , thanks and cheers to you for working on adding this model. As you have mentioned that :

> the only thing that needs to done is perhaps make [this function](https://github.com/NielsRogge/transformers/blob/5199d3d3a08264f1b17442504559c28304ce619c/src/transformers/models/h3/modeling_h3.py#L139) more like the other Attention classes in the library (like [this class](https://github.com/huggingface/transformers/blob/17a55534f5e5df10ac4804d4270bf6b8cc24998d/src/transformers/models/llama/modeling_llama.py#L158)).

So, if you do not have the spectrum as of now for the remaining task then I would love to contribute and help you out with it. :100: 
What do you think?!!

Thanks :smile:
Fix failing windows
Author: LysandreJik
Pull request: yes

Fixes an encoding issue on Windows following the merge of the import structure refactor

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33436). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

verified here that it was fixing what it intends to fix: https://github.com/huggingface/trl/pull/2051

Merging with offline approval from @ArthurZucker 

> Fixes an encoding issue on Windows following the merge of the import structure refactor
release new ver.？


@tqangxl, the merge of the import structure was not in a release, so expediting this fix in a release doesn't necessarily make sense.
Docs - update formatting of llama3 model card
Author: MichaelCurrin
Pull request: yes

# What does this PR do?

Fixes the formatting so the codeblocks inside bullet points are indented too, not breaking the flow of bullet points.

Also moved the "Note that" piece to be inline with one bullet point and code snippet instead of alone and breaking the flow.

Original:
![image](https://github.com/user-attachments/assets/0ee192e5-508d-4065-a285-81d0c1060bcd)

Tested on MDXJS playground:

![image](https://github.com/user-attachments/assets/3498a004-de4b-4740-a69e-1a7e501e13a2)
## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
`Zero-shot object detection` documentation sentence rephrase
Author: sergiopaniego
Pull request: no
Labels: name: Documentation

In [Zero-shot object detection](https://huggingface.co/docs/transformers/tasks/zero_shot_object_detection) documentation, there is an incomplete sentence:

```
...object classification and localization heads. associate images and their corresponding textual descriptions...
```

The sentence beginning with "associate images" needs to be rephrased to improve clarity and complete the thought. 

Comments:
Clean-up deprecated code
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, we can now safely remove the attribute as it has been in deprecation for a while and I think it is not very commonly used

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33446). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Pipelines download the latest revision by default instead of the default/output revision
Author: ankane
Pull request: no
Labels: name: Core: Pipeline

name: bug

### System Info

- `transformers` version: 4.44.2
- Platform: macOS-14.6.1-arm64-arm-64bit
- Python version: 3.12.5
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.3.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: no

### Who can help?

@Narsil 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import pipeline

classifier = pipeline("sentiment-analysis")
classifier("We are very happy to show you the 🤗 Transformers library.")
```

outputs

```text
No model was supplied, defaulted to distilbert/distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b (https://huggingface.co/distilbert/distilbert-base-uncased-finetuned-sst-2-english).
```

However, it downloads the `main` revision instead of `af0f99b`.

From what I can tell, this happens with all of the pipelines.

Edit: I believe it was broken in dc9147ff362e4e69829f64d28178c77cab4bef6f shortly after being added (e4d2588573f2c68eb792f2d11f092eb2c562bef5)

### Expected behavior

The downloaded revision is consistent with the output.

Adding `revision` to `hub_kwargs`  seems to fix it, but it'd probably be good to update the defaults to the latest version of each model (since that's the current behavior).

```diff
--- a/src/transformers/pipelines/__init__.py
+++ b/src/transformers/pipelines/__init__.py
@@ -860,6 +860,7 @@ def pipeline(
             f" {revision} ({HUGGINGFACE_CO_RESOLVE_ENDPOINT}/{model}).\n"
             "Using a pipeline without specifying a model name and revision in production is not recommended."
         )
+        hub_kwargs['revision'] = revision
         if config is None and isinstance(model, str):
             config = AutoConfig.from_pretrained(model, _from_pipeline=task, **hub_kwargs, **model_kwargs)
             hub_kwargs["_commit_hash"] = config._commit_hash
```

Comments:
Thanks @ankane! Would you like to open a PR to fix this?

Just opened #33395.
Revive AMD scheduled CI
Author: ydshieh
Pull request: yes

# What does this PR do?

The AMD (scheduled) CI is broken due to the several changes made in other workflow files:

- An missing part in #30266 (input argument not added but used in the caller)
- We also need to take into account the fact of having > 256 jobs (which is done for the scheduled CI but not for AMD until now)

cc @mht-sharma


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33448). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Allow send `SSH into runner` info. to DM
Author: ydshieh
Pull request: yes

# What does this PR do?

Easier to get notified.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33346). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ArthurZucker  @LysandreJik 

Thanks for the review. Sure I will ping him.
Disable `multi-gpu` jobs for now
Author: ydshieh
Pull request: yes

# What does this PR do?

Disable `multi-gpu` jobs for model jobs in daily CI.

### Context

Since 2 weeks, we can't receive the CI reports as the workflow runs failed to finish. There were remaining jobs waiting for runners. After a deeper look, they were all `multi-gpu` jobs.

@glegendre01 mentioned he is working on new runners, but there are still some `/mnt/cache` issues.

In the meantime, it's best for us to disable those `multi-gpu` jobs so we can still have some reports regarding `single-gpu` jobs.


example run page: https://github.com/huggingface/transformers/actions/runs/10576188096



I will revert #33136 before merge.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33160). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@ydshieh Do we know why they're not running? e.g. is it OOM, running too long, etc.? 

No. Some jobs could not find runners (or after waiting quite long) to run even if I didn't see any other jobs occupy those specified runner labels. (So it's more infra stuff.)

(some jobs indeed take long to run the tests - counting after once getting a runner, but that's another issue)

Link to an [internal thread](https://huggingface.slack.com/archives/C01NE71C4F7/p1724831119065499?thread_ts=1724157181.387969&cid=C01NE71C4F7)

Hi @amyeroberts I think we will need @glegendre01 to investigate this when he has time. In the meantime, it's probably better if we can have at least the single gpu jobs running and having their reports. There are indeed something we have to fix (some jobs take 3 hours to run and even expire in 6 hours etc.)
[Don't merge] gemma2
Author: ydshieh
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33204). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix flax whisper tokenizer bug
Author: hannan72
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes Bug when using whisper tokenizer for flax whisper model, according to the issue #32936


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ArthurZucker 
@sanchit-gandhi 


Comments:
Any other points @amyeroberts ?

Is it ready to merge  @amyeroberts?

@hannan72 Thanks for iterating - change now looks OK - final thing is to add a test, which would fail on current `main` but passes with this fix

@amyeroberts Test codes has been added and passed by automatic tests. Code you please do the final review?

Thanks @amyeroberts for your suggestion! It is applied and tests are split up. Just needs your approval.

> Thanks for iterating and adding tests!
> 
> General comment that unrelated formatting changes should be removed from the diff. Once the tests are split up we should be good to go

Anything else?

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33151). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts All changes has been made. I'd appreciate it if you merge the PR!
Initializes generators in trainer.py with the device specified in sel…
Author: katherine-atwell
Pull request: yes

…f.args.device

Initializes generators in Sampler and DataLoader classes with the device specified in TrainingArguments. Currently, the generators are initialized on the CPU by default, leading to errors when running Trainer on a non-CPU device.

Fixes #31897

@muellerzr @SunMarc @amyeroberts 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_31895). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Hello @muellerzr, thank you for your response! I tagged the issue that was closest to what I was experiencing, but was actually experiencing a different issue that this PR fixes. I just reported an issue reflecting what was happening (#31897) and updated the PR description to add this issue (and remove the previous issue). On my end, this fixes the issue I was experiencing, but let me know if it does for you, and if it introduces any other problems.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I just wanted to mention that I had a similar issue (`Trainer._get_train_sampler` was initialized on the wrong device, causing  a similar `RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'`) and this PR fixed it. I admittedly didn't test every HF model to verify that it doesn't break other things, though.
Qwen2-VL: clean-up and add more tests
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

This PR adds standardization on Qwen2-VL processors and adds tests for processing and generation. One thing to note is that generation tests currently will operate on text-only modality. I will add support of multimodal tests very soon

TODO:
- [x]  add video processor tests

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33354). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Tokenization in C++
Author: cnapun
Pull request: no

Is there any general strategy for tokenizing text in C++ in a way that's compatible with the existing pretrained `BertTokenizer` implementation?
I'm looking to use a finetuned BERT model in C++ for inference, and currently the only way seems to be to reproduce the `BertTokenizer` code manually (or modify it to be compatible with torchscript). Has anyone come up with a better solution than this?

Comments:
You should wait a few days if you can because @n1t0 is working on something that will very likely solve your problem and it should be ready for a first release before the end of the year.

Any update on this? It is already beyond "the end of the year".

I also tried to figure out an alternative beyond manual tokenizer. Will your approach handle with multiple models? I'm looking for a GPT-2 tokenizer in C++.

Check out this repo: https://github.com/huggingface/tokenizers

You can already use it from transformers, using `BertTokenizerFast`

Why was this closed? https://github.com/huggingface/tokenizers offers no C++ solution other than developing a Rust -> C++ interop wrapper yourself, which wouldn't work in my case.

following

This is still not available. 

We will not develop a C++ implementation of tokenizers. In case you would like C++ bindings for the `tokenizers` library, I recommend commenting on this issue dedicated to it instead: https://github.com/huggingface/tokenizers/issues/185

https://github.com/wangkuiyi/huggingface-tokenizer-in-cxx/ I built the C++ version. It works on my macOS and iPhones.

> https://github.com/wangkuiyi/huggingface-tokenizer-in-cxx/ I built the C++ version. It works on my macOS and iPhones.

Thank you for sharing, this is exactly what I needed.

sharing a Nice work. https://github.com/mlc-ai/tokenizers-cpp

I am looking for C++ implementation of tokenizer used in this model 
https://github.com/kuprel/min-dalle
Can anybody comment is it similar to hugging face tokenizer?

Hi, I have the same issue about tokenization in C++.
Is there any official way to implement?
Or maybe tokenization is not suitable for C++ application?
Any workaround for this?
cannot import BitsAndBytesConfig from transformers
Author: HWT-WalterHu
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.45.0.dev0
- Platform: Linux-3.10.0-1062.9.1.el7.x86_64-x86_64-with-glibc2.35
- Python version: 3.10.14
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.35.0.dev0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A800-SXM4-80GB

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction
I installed transformers from source code. 

from transformers import AutoModel, AutoTokenizer, BitssAndBytesConfig

got :
ImportError: cannot import name 'BitssAndBytesConfig' from 'transformers' (/opt/conda/lib/python3.10/site-packages/transformers-4.45.0.dev0-py3.10.egg/transformers/__init__.py)

### Expected behavior

what is the problem and how to fix it?

Comments:
stupid spelling mistake in the test code: BitssAndBytesConfig
Broken accuracy on LLaMa 3.1 70B -- worse than even 8B 
Author: pseudotensor
Pull request: no
Labels: name: bug

### System Info

transformers==4.43.1

As built into vLLM or TGI as well.

4*H100 for llama 3.1 70B or 1*H100 for llama3.1 8B

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

https://github.com/vllm-project/vllm/issues/6760

### Expected behavior

I'd expect long-context performance to not be worse for 70B llama 3.1 compared to 8B, and for the answers to be "normal" and not so brief.

Comments:
Hey! Not super sure what's going on as your issues seems to show usage of VLMM, which does not use transformers as a backend (yet? 👀 just joking) 

The same happens with TGI as I mentioned, so I assumed was general to all code that gets pulled from transformers ultimately.  I have yet to run transformers directly, but can.

It's possible it's just a model issue, but a bit surprising that Meta would mess things up so badly.

When prior llama's came out, there were similar issues with things being off somewhere in handling of the model or tokenizer.

It can be versioning as well, the ROPE scaling is super important! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.