noamaanMulla-03/transformers-issues huggingface dataset:
RuntimeError: slow_conv2d_forward_mps: input(device='cpu') and weight(device=mps:0') 
Author: yeseliushi
Pull request: no
Labels: name: Examples

name: Vision

### System Info

- `transformers` version: 4.42.0.dev0
- Platform: macOS-14.5-arm64-arm-64bit
- Python version: 3.9.6
- Huggingface_hub version: 0.23.4
- Safetensors version: 0.4.3
- Accelerate version: 0.31.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.3.1 (False)
- Tensorflow version (GPU?): 2.16.1 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

python run_image_classification.py \
    --dataset_name beans \
    --output_dir ./beans_outputs/ \
    --remove_unused_columns False \
    --label_column_name labels \
    --do_train \
    --do_eval \
    --push_to_hub \
    --push_to_hub_model_id vit-base-beans \
    --learning_rate 2e-5 \
    --num_train_epochs 5 \
    --per_device_train_batch_size 8 \
    --per_device_eval_batch_size 8 \
    --logging_strategy steps \
    --logging_steps 10 \
    --eval_strategy epoch \
    --save_strategy epoch \
    --load_best_model_at_end True \
    --save_total_limit 3 \
    --seed 1337

### Expected behavior

I don't know

Comments:
Hi @yeseliushi thanks for reporting, I will have a look!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
tokenizer.convert_ids_to_tokens(tokenizer.convert_tokens_to_ids(x)) returning a different result
Author: LeonieWeissweiler
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

I have a trained Esperanto Tokenizer, the training code was taken from the how_to_train example. I'm using Roberta Tokenizer as RobertaTokenizerFast doesn't work with trainer.py yet (or last time I checked).

```python
from transformers import RobertaTokenizer

alt_tokenizer = RobertaTokenizer.from_pretrained("./drive/My Drive/EsperBERTo", max_len=512)

alt_tokenizer.convert_ids_to_tokens(alt_tokenizer.convert_tokens_to_ids(alt_tokenizer.tokenize('Neniu atendas la Hispanan Inkvizicion.')))

```
output is: ['Neniu', 'Ġatendas', 'Ġla', 'ĠHispan', 'an', 'ĠIn', 'kvizi', 'cion', '.']

## Expected behavior

the same output as input

## Environment info
tokenizers               0.8.0rc1       
transformers             2.11.0   
     
- `transformers` version: 2.11.0
- tokenizers version: 0.8.0rc1
- Platform: colab
- Python version: 3.6
- Using GPU in script?: No


Comments:
do u find out why
Fix tests
Author: molbap
Pull request: yes

# What does this PR do?

This PR tracks and should fix current failing tests on `main`. They were introduced in https://github.com/huggingface/transformers/pull/32493, which was green before merging - this current PR uses `no-filter` to ensure all tests pass.

cc @amyeroberts 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32649). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Only tests failing without any filtering on the fetcher are tensorflow-related 
- test_examples_tensorflow.py has a zip file error
- tests/models/swiftformer/test_modeling_swiftformer.py::SwiftFormerModelTest::test_pt_tf_model_equivalence fails the equivalence check by a little
perhaps cc @Rocketknight1 if that's of relevance?

For current failing test tests/models/mobilenet_v2/test_modeling_mobilenet_v2.py::MobileNetV2ModelTest it is just flaky as flagged. Rerunning with filters on and should be OK to merge
Update beam_constraints with KMP
Author: threeFeetCat123
Pull request: yes
Labels: name: Generation

# What does this PR do?

This PR enhances the `PhrasalConstraint` class with the Knuth-Morris-Pratt (KMP) algorithm. This allows the constraint to restart quickly rather than start from the beginning when a match fails, improving efficiency.

Here is test:
```
from transformers import AutoTokenizer
import beam_constant


def test():
    tokenize = AutoTokenizer.from_pretrained("Qwen-7B-Chat", trust_remote_code=True)
    force_word = " love cat, love dog"
    force_word_id = tokenize.encode(force_word, add_special_tokens=False)
    constraint = beam_constant.PhrasalConstraint(force_word_id)
    generate_contest = "I love cat, love cat, love dog"
    generate_tokens = tokenize.encode(generate_contest, add_special_tokens=False)
    print(force_word_id)
    print(constraint.match_table)
    for token_id in generate_tokens:
        step, complete, reset = constraint.update(token_id)
        print(f"token_id:{token_id} {constraint.fulfilled_idx} {step} {complete} {reset}")

if __name__ == "__main__":
    test()
```

test result

![image](https://github.com/huggingface/transformers/assets/124332948/9d83ad73-5087-49b3-a93d-a5802c198b1d)



## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?
## Who can review?

@gante


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @threeFeetCat123 👋 

Thank you for the PR! We're pausing contributions to constrained beam search while we refactor beam search to have `torch.compile` support, will revisit this PR after that is complete :)

Thank you for your reply
Fix tests recurrent
Author: molbap
Pull request: yes

# What does this PR do?

Continuation of #32649 since it seems RecurrentGemma tests failures were not caught

Comments:
Only the tensorflow-examples are failing here, again, so let's see? 
Not re-triggering the CI with the filtering, so people can take a look more easily if wanted 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32651). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@molbap Are there any previously passing tests which now are being skipped with this change? 

I don't think there are - if `position_ids` were absent in the signature, this test would indeed have failed

Just checked, only additional tests skipped are those coming from RecurrentGemma as far as I can tell
In previous PR:
```bash
================ 2843 passed, 5777 skipped in 111.91s (0:01:51) ================
```
now:
```
=========== 1 failed, 2846 passed, 5773 skipped in 108.18s (0:01:48) ===========
```
That's on one out of 5 parallel runs, of course
Interpolate clip
Author: manuelsh
Pull request: yes

This PR addresses the suggestions of @amyeroberts in the existing PR https://github.com/huggingface/transformers/pull/30783.




Comments:
@amyeroberts I've implemented changes in all clip family models to account for the use case when `interpolate_pos_encoding=False` and the resolution is not the same as the pretrained model. I also implemented  its respective tests.

I've attempted to sync with the huggingface:main branch with 
```
git fetch upstream
git rebase upstream/main
```
as there are some failed tests due to being out of sync, but is not working. 

When running  `make repo-consistency` I get:

```
python utils/check_copies.py
Traceback (most recent call last):
  File "/usr/src/app/transformers/utils/check_copies.py", line 1106, in <module>
    check_copies(args.fix_and_overwrite, args.file)
  File "/usr/src/app/transformers/utils/check_copies.py", line 852, in check_copies
    new_diffs = is_copy_consistent(filename, overwrite, buffer)
  File "/usr/src/app/transformers/utils/check_copies.py", line 675, in is_copy_consistent
    target_lines, theoretical_code, theoretical_code_splits = find_code_and_splits(
  File "/usr/src/app/transformers/utils/check_copies.py", line 521, in find_code_and_splits
    code, (lines, target_start_index, target_end_index) = find_code_in_transformers(
  File "/usr/src/app/transformers/utils/check_copies.py", line 456, in find_code_in_transformers
    raise ValueError(f" {object_name} does not match any function or class in {module}.")
ValueError:  models.clip.test_modeling_clip.CLIPModelTest.test_model_get_set_embeddings does not match any function or class in models/clip/test_modeling_clip.
```
and I see I am missing this test and the get_set_embeddings even after fetching.

@amyeroberts help would be appreciated.

Closing this one in favour of #32600
TF_Deberta supporting mixed precision
Author: pinesnow72
Pull request: yes

# What does this PR do?

This PR fixes some codes of TF DeBERTa (version 1 and 2) that do not support Mixed Precision.

With TF and Keras, mixed precision is configured with the following settings:

```
policy = keras.mixed_precision.Policy('mixed_float16')
keras.mixed_precision.set_global_policy(policy)
```

However, the existing TFDebertaModel and TFDebertaV2Model throws TypeError with this mixed precision setting.
With mixed precision, TF and Keras requires to use Layer.dtype for model or layer's weights and Layer.compute_dtype for internal tensor computation. But the current TFDebertaModel and TFDebertaV2Model codes do not seem to reflect this requirement and definitely assume the dtype would be tf.float32.

I searched and corrected some error-prone code snippets in modeling_tf_deberta.py and modeling_tf_deberta_v2.py.
This correction worked correctly with mixed precision in my local machine.

This PR fixes [#31989](https://github.com/huggingface/transformers/issues/31989)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [V] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [V] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@Rocketknight1, @ArthurZucker 


Comments:
Hi @pinesnow72, this looks good! Feel free to ping me whenever it's ready for review. The issues with code quality can be fixed by doing `pip install transformers[quality]` followed by `make style` in the repo directory.

> Hi @pinesnow72, this looks good! Feel free to ping me whenever it's ready for review. The issues with code quality can be fixed by doing `pip install transformers[quality]` followed by `make style` in the repo directory.

Hi @Rocketknight1, As your comment, the style was corrected and uploaded. Thanks.

LGTM now! cc @amyeroberts for core maintainer review, and thanks for the PR!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32618). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks for accepting this PR
`LlamaTokenizer` refactor tests.
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Increase the coverage of our tests

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30879). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix video batching to videollava
Author: merveenoyan
Pull request: yes

cc @zucchini-nlp works with no problems

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32139). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp one thing confused me, you are passing numpy tests but don't pass `numpify=True` when creating dummy inputs (also they pass the assertion of being numpy arrays) I couldn't make any change to the numpy test to make it for PIL

Yes, I made bad decision when constructing tests tailoring only for torch and numpy. This one has better dummy video preparation, can copy from it maybe
 
https://github.com/huggingface/transformers/blob/3aefb4ec7f957f9561a410eabc6f9d57b2f0384f/tests/models/llava_next_video/test_image_processing_llava_next_video.py#L99-L124
fix: Fixed failing tests in `tests/utils/test_add_new_model_like.py`
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
Fixed, the following failing tests in `tests/utils/test_add_new_model_like.py`
```
tests/utils/test_add_new_model_like.py::TestAddNewModelLike::test_get_model_files
tests/utils/test_add_new_model_like.py::TestAddNewModelLike::test_get_model_files_only_pt
tests/utils/test_add_new_model_like.py::TestAddNewModelLike::test_get_model_files_tf_and_flax
tests/utils/test_add_new_model_like.py::TestAddNewModelLike::test_retrieve_info_for_model_with_vit
tests/utils/test_add_new_model_like.py::TestAddNewModelLike::test_retrieve_info_for_model_with_wav2vec2
tests/utils/test_add_new_model_like.py::TestAddNewModelLike::test_retrieve_model_classes
```
Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@amyeroberts @LysandreJik 

Comments:
Fix generate with `inputs_embeds` as input
Author: molbap
Pull request: yes
Labels: name: run-slow

# What does this PR do?

Seems recent refactor broke generation based on `inputs_embeds` if I understand correctly.
This aims at catching correctly the `sequence_length` by using the passed `model_inputs`, creating the correctly-sized `causal_mask` in generation step. 

Core change is 

```python
        if inputs_embeds is not None and cache_position[0] == 0:
            model_inputs = {"inputs_embeds": inputs_embeds, "input_ids": None}
        else:
            # The clone here is for the same reason as for `position_ids`.
            model_inputs = {"input_ids": input_ids.clone(memory_format=torch.contiguous_format), "inputs_embeds": None}
        # or any Cache class
        if isinstance(past_key_values, HybridCache) and attention_mask.ndim == 2:
            if model_inputs["inputs_embeds"] is not None:
                batch_size, sequence_length, _ = model_inputs["inputs_embeds"].shape
                device = model_inputs["inputs_embeds"].device
            else:
                batch_size, sequence_length = model_inputs["input_ids"].shape
                device = model_inputs["input_ids"].device
```



Fixes https://github.com/huggingface/transformers/issues/32479

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32493). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Yes investigating that to add a test, a bit surprised that this slipped through as we're supposed to be testing this already :thinking: 

Hey @molbap, I think that we forgot to skip a few tests related to this PR : [test_1](https://app.circleci.com/pipelines/github/huggingface/transformers/100558/workflows/0c640751-f999-4b6a-9f8d-ebc7f869c50f/jobs/1339096?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-checks-link&utm_content=summary) and [test_2](https://app.circleci.com/pipelines/github/huggingface/transformers/100558/workflows/0c640751-f999-4b6a-9f8d-ebc7f869c50f/jobs/1339087) . Can you confirm me that we indeed need to skip them just like bert model ? I'll open a PR if that's the case ! Not sure why these tests are only triggered on my PR: https://github.com/huggingface/transformers/pull/32627

Hi @SunMarc , there was an issue with the fetching of tests and it is causing a lot of failures currently, opening a PR to fix it - in the meantime you can indeed skip them!

In the end #32651 was also needed to fix breaking tests introduced here, FYI!
Support loading shard GGUF models
Author: Isotr0py
Pull request: no
Labels: name: Feature request

### Feature request

- Support loading from sharded GGUF model files (For example, [legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF](https://huggingface.co/legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF/tree/main)).

### Motivation

- For some large models, the GGUF format model will be split into multiple shards during converting. 
- However, the GGUF integration in `transformers` only supports single file GGUF model currently.

### Your contribution

- This kind of GGUF model has been split into multiple shards, formatted as `<ShardNum>-of-<ShardTotal>`, just like sharded `safetensors`. 
- The header for config and tokenizer handing are kept in the first shard file (For example, [Meta-Llama-3.1-70B-Instruct.Q5_K-00001-of-00003.gguf](https://huggingface.co/legraphista/Meta-Llama-3.1-70B-Instruct-IMat-GGUF/blob/main/Meta-Llama-3.1-70B-Instruct.Q5_K/Meta-Llama-3.1-70B-Instruct.Q5_K-00001-of-00003.gguf)).
- For more details about sharded GGUF files, please refer to [gguf.md](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md).

Comments:
Ah very cool! Thanks foir sharing @Isotr0py, it would be nice to indeed have support for this sharding.

If you'd like, we're very open to PRs :)

cc @SunMarc 

maybe cc @phymbert for visibility too

and linking the related https://github.com/ggerganov/llama.cpp/issues/9023
Fix `JetMoeIntegrationTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

They are failing when using `device_map=auto`. It seems to related to the usage of `torch.empty`

> self.bias = torch.nn.Parameter(torch.empty(self.input_size))

The error is

> (line 740)  RuntimeError: Tensor on device meta is not on the expected device cuda:0!

run page is 

https://github.com/huggingface/transformers/actions/runs/10137089326/job/28056567474


I think it would still be good if @SunMarc  @muellerzr to have a look on why using `auto` fails here. But this PR just remove `auto`.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32332). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update the distributed CPU training on Kubernetes documentation
Author: dmsuehir
Pull request: yes

# What does this PR do?

This PR updates the distributed CPU Kubernetes example to use a new container (python 3.10.12, torch 2.3.0+cpu, IPEX 2.3.0, Transformers 4.44.0) and updates the PyTorchJob yaml. I changed the PyTorchJob yaml to use a bash script in the args, which makes it easier to copy/paste in commands and also allows more flexibility to show how installing requirements.txt and sourcing the Intel oneCCL Bindings for PyTorch setvars.sh can be done as part of the Kubernetes job. I removed `namespace` from the yaml and instead specified the namespace in the `kubectl` commands. The cpu/node updates reflect an example of using 6th generation Xeon (granite rapids).

I tested building the container and running it on a Kuberntes cluster with Xeon nodes. I've also tested the doc build using `doc-builder build`.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@stevhliu


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_32669). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
chore: move `conftest.py` to `tests/`
Author: SauravMaheshkar
Pull request: no
Labels: name: Feature request

name: WIP

### Feature request

It's a common practice to place the `conftest.py` file within the `tests/` directory.


### Your contribution

Initial Attempt: https://github.com/huggingface/transformers/pull/32011

### References

* https://github.com/pytest-dev/pytest/discussions/10708

Comments:
cc @ydshieh