noamaanMulla-03/transformers-issues huggingface dataset:
Add support for Pixtral
Author: ArthurZucker
Pull request: yes
Labels: name: New model

# What does this PR do?

Pixtral is basically LLAVA with a new vision encoder that uses ROPE. 

Closes https://github.com/huggingface/transformers/issues/33441




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33449). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
add `private` parameter to trainer.push_to_hub
Author: not-lain
Pull request: no
Labels: name: Feature request

### Feature request

Adding another parameter to the `trainer.push_to_hub` called `private`

### Motivation

a user at https://discuss.huggingface.co/t/how-to-push-trained-models-and-save-it-privately-on-hub/25538/10 has requested this, and although setting `hub_private_repo` training argument to True is a good alternative, but I agree with them on the basis that we need a unified `push_to_hub` method.

### Your contribution

I can submit a PR 

Comments:
@not-lain thanks for making this. Let me know if you want me to do this or if you want any help :)

Honestly, I'm a bit overwhelmed lately and I could use some help.
Feel free to open a PR and tag me and I'll see if I have any reviews.
I was thinking of doing something like this to ensure backward compatibility 

```python
def push_to_hub(...,private=None,...) : 
   if private is not None : 
       self.args.hub_private_repo = private
  
```

but feel free to ignore this and build it your own way.
Thanks in advance
chore: fix typo in comment in tokenization_utils_base.py
Author: DavidLemayian
Pull request: yes

# What does this PR do?

Small typo fix in tokenization_utils_base.py comment


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@stevhliu 


Comments:
Optimised 4bit inference kernels
Author: nivibilla
Pull request: no
Labels: name: Feature request

### Feature request

Integration of new 4bit kernels 

https://github.com/IST-DASLab/marlin


### Motivation

provide faster Inference than awq/exllama for batch sizes upto 32


### Your contribution

Just saw this today, can try provide sample notebook. 

Comments:
cc @younesbelkada @SunMarc 

Thanks !
@efrantar can confirm but looking at the code it looks like you essentially just need to replace all Linear layers with `marlin.Linear` ? (not 100% sure) if the interface is simple we can definitely add that support I think by passing `MarlinConfig` through `quantization_config` in `from_pretrained`.

We also do have HQQ in the backlog https://github.com/huggingface/transformers/issues/28328 but we are waiting to finalize https://github.com/huggingface/transformers/pull/26610 from @poedator before adding any new quantization scheme

cc @Titus-von-Koeller just FYI

@qwopqwop200 seems to be working on adding marlin to AutoGPTQ. If it is merged, we will also have support with transformers quite easily. https://github.com/qwopqwop200/AutoGPTQ-add-marlin

Yes, replacing the layers is pretty much it. It might also be possible to write a (not too complex) kernel to convert a GPTQ format model (groupsize 128, sym, no act-order; or any other quantization method that produces such models) to Marlin format on-the-fly (when loading the model) in reasonable time, which could be useful to have only a single storage format. However, I am not sure how many of the current GPTQ models on the hub already use the required settings for Marlin.

Thank you very much @efrantar for the precision! We will update you as soon as we merge #26610 

Any update for this feature?

I will have a look at it soon ! Since it is available on autogptq, the integration should be straightforward ! 

https://github.com/ist-daslab/sparse-marlin

Any update?
llama model can't generate EOS
Author: ZhangMaoTai
Pull request: no

### System Info

python                    3.8.16
torch                     1.13.1
transformers              4.28.0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig

# both model have same behavior
model_path = "luodian/llama-7b-hf"
# model_path = "huggyllama/llama-7b"

model = LlamaForCausalLM.from_pretrained(model_path)
tokenizer = LlamaTokenizer.from_pretrained(model_path)
tokenizer.pad_token = tokenizer.eos_token

text = ["Translate english to chinese: I love you.", "What is your name:"]
a = tokenizer(text, return_tensors='pt',padding="longest")
print(model.generate(**a, max_new_tokens=64))
```

### Expected behavior

The llama model's generate method doesn't generate any EOS token under any circumstances

Comments:
cc @ArthurZucker 

Hey!
This seems to be a bit similar to #23175. 
When the `generate` function is called, it should stop once the `eos_token` (which is `2`). 
If the model does not predict it, then the generate function will not stop. This can come from the training, but is most probably not an issue with the `generate` function. 

You can check the original behaviour here: https://github.com/facebookresearch/llama/blob/main/llama/generation.py you'll see that it does not stop on the `eos` token. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

what was the soln?

Same issue

The model does not predict it with the generation parameters that are used. 

Same... I'm using [unsloth llama-3-base](https://colab.research.google.com/drive/1XamvWYinY6FOSX9GLvnqSjjsNflxdhNc?usp=sharing#scrollTo=QmUBVEnvCDJv)

> Same... I'm using [unsloth llama-3-base](https://colab.research.google.com/drive/1XamvWYinY6FOSX9GLvnqSjjsNflxdhNc?usp=sharing#scrollTo=QmUBVEnvCDJv)

Hey, did you find any solution for llama 3 unsloth. I've fine-tuned the model with tokenizer.eos_token at the end. However, my model is not able to predict the eos token at all. It'll generate a coherent response then start spamming the last sentence again and again till hitting max length.
Cohere: update RoPE structure
Author: gante
Pull request: yes

# What does this PR do?

This PR propagates the updates to the RoPE structure to `cohere` -- the logic for RoPE was abstracted into a separate module for `llama3.1` (#32135). Using the new structure, a model has access to all RoPE scaling strategies.

While touching the modeling code, I've taken the liberty to:
1. update `copied from` statements, which were disabled in previous PRs;
2. update (postpone) deprecation messages to ensure deprecated features are removed from all models in the same version.

✅ all slow tests passing

_________________________________
Note: #31999 was originally open to migrate all modern RoPE models into the upgraded structure. However, working on `cohere`, I noticed that there may be important implementation differences in RoPE. As such, I'll be opening multiple PRs, batching similar RoPE implementations together.


Comments:
@LysandreJik a PR like this one will be open for a few more modern models. Since part of the changes consists of having a global view of the model to update the `copied from` statements, would you like me to update the import structures as well? 🤗 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33408). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> @LysandreJik a PR like this one will be open for a few more modern models. Since part of the changes consists of having a global view of the model to update the copied from statements, would you like me to update the import structures as well? 🤗

No need to updat the import structure for now!
Fix SSH
Author: ydshieh
Pull request: yes

# What does this PR do?

Fix

GitHub' secret can't contain `-` character. But we have @zucchini-nlp 

This PR does a replacement of `-` by `_`

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33451). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add keypoint-detection task guide
Author: merveenoyan
Pull request: yes

I added task guide for the recent keypoint detection task and SuperPoint model. I also attempted to add feature matching with this model but apparently this model doesn't do that so it only demonstrates keypoint detection. Later when SuperGlue is merged I will add feature matching.

Comments:
IDK why the image is broken, it's here https://huggingface.co/datasets/huggingface/documentation-images/blob/main/bee.jpg

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33274). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Whisper] Fix generate and tokenizer behavior with added tokens
Author: eustlb
Pull request: yes

# What does this PR do?

Fixes #33082. 

### Description of the issue

Transformers' `PretTrainedTokenizer` [`_add_tokens`](https://github.com/huggingface/transformers/blob/ce62a41880b5b70a304d068eb58f55894a5a7af8/src/transformers/tokenization_utils.py#L510) add tokens at the end of the vocabulary. Likewise, `PreTrainedModel`'s [`_get_resized_embeddings`](https://github.com/huggingface/transformers/blob/ce62a41880b5b70a304d068eb58f55894a5a7af8/src/transformers/modeling_utils.py#L2080) will add newly initialized tensors at the end. This behavior is not compatible with Whisper's timestamp token identification. Indeed, official implementation as well as Transformer's one identifies timestamp tokens as the ones that have an id > `timestamp_begin`. For this reason, the newly added tokens are falsely considered timestamps and this poses decoding issues. 

###  Implementation choices

Two possibilities here: 
1. change Whisper's decoding logic using a `timestamp_end`
2. overwrite  [`_add_tokens`](https://github.com/huggingface/transformers/blob/ce62a41880b5b70a304d068eb58f55894a5a7af8/src/transformers/tokenization_utils.py#L510)  and [`_get_resized_embeddings`](https://github.com/huggingface/transformers/blob/ce62a41880b5b70a304d068eb58f55894a5a7af8/src/transformers/modeling_utils.py#L2080) in order to add new tokens before the first timestamp token (and not at the end as current implementation)

I chose option 1 to avoid overwriting the default methods, focusing instead on modifying the Whisper-specific generation method and tokenizer logic.

## Who can review?

@ylacombe 



Comments:
[Whisper test] Fix some failing tests
Author: ylacombe
Pull request: yes
Labels: name: run-slow

# What does this PR do?

This PR proposes multiple simple fix to some Whisper failing tests:
1. Fix failing tensor device
2. Fix long-form integration tests that weren't using `return_timestamps=True`

UPDATE: ready for review, I'll correct more tests in the coming days 

cc @LysandreJik and @ydshieh 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33450). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

It would be a good opportunity to use the `run-slow` in this PR 

There'll be failing tests though FYI, there's a lot of work to do around Whisper!

Yeah I understand. (just want to make sure the tests fixed in this PR are passing).

The core maintainers could still merge in that case.

@ydshieh, do you know why this error happens ?
`Bus error (core dumped)`


No, it happens in some models and runs. It's more a infra issue I think. 

I re-triggered it. Let's see

Still happening!

Let me see what I can do manually.

I pushed a commit to run slow ci on the (about to deploy) new cluster. Let me know how it goes 🙏 

(before merge, let me revert my commits 🙏 )

Could you check if the fixed tests are passing in the following run ?


> https://github.com/huggingface/transformers/actions/runs/10885416563/job/30203010247?pr=33450 

or download the artifact

> https://github.com/huggingface/transformers/actions/runs/10885416563?pr=33450

and check the file `summary_short.txt` would be easier

Hey @ydshieh, they passed indeed ! Thanks for helping me on this,
I'll merge once you revert the commit!

Nice! Reverted!
TypeError: MistralForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'
Author: edchengg
Pull request: no
Labels: name: bug

### System Info

I got this error when I tried to use sentiment classification pipeline with  "nvidia/Mistral-NeMo-Minitron-8B-Base". It works fine with llama 3.1.

TypeError: MistralForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'
- transformers              4.43.4   
- 
See code below:

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```
import torch 
from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline 

torch.random.manual_seed(0) 

model_id = "nvidia/Mistral-NeMo-Minitron-8B-Base" # not working
# model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct" # working
model = AutoModelForSequenceClassification.from_pretrained( 
    model_id,  
    device_map="auto",  
    torch_dtype=torch.bfloat16 
) 

tokenizer = AutoTokenizer.from_pretrained(model_id) 



pipe = pipeline( 
    "sentiment-analysis", 
    model=model, 
    tokenizer=tokenizer, 
) 


output = pipe("hello how are you today?")
print(output)
# TypeError: MistralForSequenceClassification.forward() got an unexpected keyword argument 'token_type_ids'

```

### Expected behavior

expected to be able to run the code

Comments:
Thanks for your PR @edchengg! Does it work with the default 7b mistral model on your side? So for example with this checkpoint: `mistralai/Mistral-7B-Instruct-v0.1`

(You either need to delete the `token_type_ids`, or update `tokenizer.model_input_names` 🤗
[i18n-ar] Translated file : `docs/source/ar/run_scripts.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/run_scripts.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
thank you @abodacs
[i18n-ar] Translated file : `docs/source/ar/preprocessing.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/preprocessing.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
Thank You @abodacs
[i18n-ar] Translated file : `docs/source/ar/autoclass_tutorial.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/autoclass_tutorial.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
Thank you @abidlabs
[i18n-ar] Translated file : `docs/source/ar/llm_tutorial.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/llm_tutorial.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments: