noamaanMulla-03/transformers-issues huggingface dataset:
The chatml template doesn't have a system message in the tokenizer_config.json
Author: snehith-r
Pull request: no

### System Info

I don't see any system message in the chatml template of tokenizer_confi.json but the example in the docs gives system message as well in the messages,is this a mistake does phi3-instruct doesn't take a system message? even when I tried the same example give here https://huggingface.co/docs/transformers/main/model_doc/phi3 I don't see any system tags in the output.

"chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') %}{{'<\|user\|>' + '\n' + message['content'] + '<\|end\|>' + '\n' + '<\|assistant\|>' + '\n'}}{% elif message['role'] == 'assistant' %}{{message['content'] + '<\|end\|>' + '\n'}}{% endif %}{% endfor %}"


### Who can help?

@ArthurZucker @youne

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

messages = [{"role": "system", "content": "You are a helpful digital assistant. Please provide safe, ethical and accurate information to the user."},{"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

outputs = model.generate(inputs, max_new_tokens=32)
text = tokenizer.batch_decode(outputs)[0]
print(text)
Output:


 
Can you provide ways to eat combinations of bananas and dragonfruits?
 

 
Certainly! Bananas and dragonfruits can be combined in various delicious ways. Here are some ideas for eating combinations of bananas and

### Expected behavior



 You are a helpful digital assistant. Please provide safe, ethical and accurate information to the user.

 Can you provide ways to eat combinations of bananas and dragonfruits?

 Absolutely! Bananas and dragonfruits are both delicious fruits that can be combined in various ways to create tasty and nutrit

Comments:
cc @Rocketknight1 

This looks valid - the model's chat template doesn't support `system` messages, but the examples show system messages, and the model does have a `
` token. Will try to grab someone who worked on the port!

Hi @snehith-r, we confirmed with Microsoft that Phi-3 was **not** trained with system messages and therefore doesn't understand them. The chat template is correct, and the examples were wrong. They should now be fixed - thanks for reporting it!
Fix CI after #30410
Author: zucchini-nlp
Pull request: yes
Labels: name: run-slow

# What does this PR do?

As per title, the prev PR forgot to replace Embedding layer in one of the models which caused red CI. This PR fixes it :)

cc @ydshieh 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30612). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@zucchini-nlp would you like to try the `run-slow` feature 😆 ?

Rebase on main to include #30540 (if not yet)

Push an empty commit with the message: `[run-slow] blenderbot`

Yes, lets see the slow-tests in action 😄 

```
========== 229 passed, 151 skipped, 73 warnings in 570.48s (0:09:30) ===========
```
(note: the above run is not the Pipeline tests where we found the issue in the first place)
Allow different classification labels for each input sequence for zero shot classification to enhance batching performance
Author: ABasharEter
Pull request: yes

# What does this PR do?
Following on this feature request [https://github.com/huggingface/transformers/issues/29793](https://github.com/huggingface/transformers/issues/29793)

This PR create a new pipeline for zero shot text classification that accept different type of input. This input type group the labels there respective sequence. With that you can do one call with multiple different labels and batch the prediction. This will make prediction more efficient in the case of multiple type of labels classification. 


## Before submitting


- [] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline] (https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. https://github.com/huggingface/transformers/issues/29793
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@younesbelkada @ArthurZucker @Narsil



Comments:
Thanks for the comments. TBH some of the work I am doing will require this change so I thought it is better to contribute this improvement. 

Also some test cases are failing and I am not sure why. Can you please give me some hits on what it should do and why it is timing out?

Also do you mean that it wont be merged now? and if so what is the best way to use this update in my project without building my custom transformers? wheel as this will be hard to maintain.

Failing tests are unrelated! 
Regarding using these, you can probably use this branch directly or just push the model to the `hub` and use `trust_remote_code=True`. Even locally you can use that! 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix BatchEncoding not prepending batch axis when tensor_type is None
Author: mig-mfreitas
Pull request: yes

# Fix BatchEncoding not prepending batch axis to python list when tensor_type is None

Fixes #28882

## Who can review?
@ArthurZucker


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
add mlp bias for llama models
Author: mayank31398
Pull request: yes

This adds bias support for MLP in Llama
Can we add this?
It would help a lot for some models we are developing
@ArthurZucker and @younesbelkada

This lets us re-use the llama model class for our models :) without adding another class for a new model.


Comments:
@younesbelkada light ping again

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30031). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Thanks everyone for the quick turnaround 🤗
Wav2Vec2ForCTC weight mismatch
Author: MahmoudAshraf97
Pull request: no

### System Info

- `transformers` version: 4.40.1
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.1+cu121 (False)
- Tensorflow version (GPU?): 2.15.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.8.3 (cpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help?

Hi everyone,
@patrickvonplaten , @sanchit-gandhi 
almost all Wav2Vec2 Models have a weight mismatch and give this error
```
Some weights of the model checkpoint at jonatasgrosman/wav2vec2-large-xlsr-53-arabic were not used when initializing Wav2Vec2ForCTC: ['wav2vec2.encoder.pos_conv_embed.conv.weight_g', 'wav2vec2.encoder.pos_conv_embed.conv.weight_v']
- This IS expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing Wav2Vec2ForCTC from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of Wav2Vec2ForCTC were not initialized from the model checkpoint at jonatasgrosman/wav2vec2-large-xlsr-53-arabic and are newly initialized: ['wav2vec2.encoder.pos_conv_embed.conv.parametrizations.weight.original0', 'wav2vec2.encoder.pos_conv_embed.conv.parametrizations.weight.original1']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```
I've tried multiple models from official and non official sources and all have the same issue with both pytorch and safetensors variant


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python

from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

MODEL_ID = "jonatasgrosman/wav2vec2-large-xlsr-53-arabic"
processor = Wav2Vec2Processor.from_pretrained(MODEL_ID)
model = Wav2Vec2ForCTC.from_pretrained(MODEL_ID)
```
the problem exists in all versions starting from this commit bc9ecef94242f3247441ec60167cb026d871280e

### Expected behavior

The model to load successfully

Comments:
After searching I found that it's a duplicate of #26796 and #27605
HQQ: PEFT support for HQQ
Author: younesbelkada
Pull request: yes

# What does this PR do?

Tests are going to be added directly into: https://github.com/huggingface/peft/pull/1618

cc @SunMarc 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30632). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add sidebar tutorial for chat models
Author: Rocketknight1
Pull request: yes

~Draft for now~ - this PR will add a sidebar tutorial on chat models!

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30401). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

This should be ready for review! cc @stevhliu. Also cc @zucchini-nlp and @gante since I'm talking about generation-related topics a lot, let me know if there's anything I'm missing, or if I should be tying this into the existing generate docs better!

@Rocketknight1 cool tutorial 🙌
Avoid duplication in PR slow CI model list
Author: ydshieh
Pull request: yes

# What does this PR do?

If there are duplicated items, the upload artifacts step would fail as shown in 

https://github.com/huggingface/transformers/actions/runs/8935717727/job/24544707905?pr=30136

> Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30634). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update pipeline torch_dtype property not to return dtype when it is not Pytorch model.
Author: B-Step62
Pull request: yes

# What does this PR do?
Follow-up for #28940. The previous PR added `torch_dtype` property, but it also returns dtype string for Tensorflow model, which is not accurate behavior for the property name. This PR fixes the property only returns `torch.dtype` object of Pytorch models.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests? Yes, also tested the property for Pytorch/Tensorflow models locally.


## Who can review?

@ArthurZucker (as the reviewer of the previous PR)

Comments:
All red tests ^ failed at `pip install -U --upgrade-strategy eager .[sklearn,torch,testing,sentencepiece,torch-speech,vision,timm]` but seems unrelated...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Urdu Encoding Issue in Hugging Face Tokenizer 
Author: Frankenstein-hsa
Pull request: no

### System Info

@ArthurZucker   i was training hugging face tokenizer on urdu dataset 
 However instead of having urdu tokens and merges in vocabulary it shows wiered encoding Here is an example:

ÚĪÛĮ': 393,
 'ĠØ³': 88,
 'ÙĪØ±': 101,
 'Ø¨Ø±': 255,
 'ĠÙħØª': 298,
 'ĠÙĪØ§ÙĦ': 249,
 'Ø³Ù¹': 305,
 'ĠÚ¯Ø§': 319,
 'ĺ': 61,
 'ĠØ¯ÙĨÛĮØ§': 324,
 '²': 20,
 'ĠØ¢': 146,
 'Ø±ÛĮ': 119,
 'Ø·': 174,
 'Ĺ': 60,
 'Ú©Ú¾': 335,
 'Ĵ': 55,
 'Ø§': 69,

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

from datasets import load_dataset

dataset = load_dataset("El-chapoo/Symboleess-urdu-1", split="train")


def get_training_corpus():
    for i in range(0, len(dataset), 1000):
        yield dataset[i : i + 1000]["cleaned_text"]

!pip install datasets
from datasets import load_dataset
from tokenizers import Tokenizer
from tokenizers.models import BPE
from transformers import AutoTokenizer

from tokenizers import (
    decoders,
    models,
    normalizers,
    pre_tokenizers,
    processors,
    trainers,
    Tokenizer,
)

tokenizer = Tokenizer(models.BPE())

tokenizer.pre_tokenizer = pre_tokenizers.ByteLevel(add_prefix_space=False)


trainer = trainers.BpeTrainer(vocab_size=400,special_tokens=["<pad>","
","
","<mask>"],
                          max_token_length=16)

tokenizer.train_from_iterator(get_training_corpus(), trainer=trainer)

tokenizer.get_vocab()



### Expected behavior

I have followed the guidelines from hugging face documentation . However gpt2  tokenizer also show same behaviour ,But Llama and Gemma tokenizer work expected but the consume alot of memory and requires much more time when training

Comments:
when we save it the issue is solved
[WIP] adds AudioMAE model
Author: ps4vs
Pull request: yes
Labels: name: New model

name: Audio

Fixes https://github.com/huggingface/transformers/issues/27453 Audio-MAE - ViTMAE for audio.

@ArthurZucker @sanchit-gandhi @ylacombe @pennychong94

There are some nit changes pending. Looking forward to push within 2-3 days.

Thanks @NielsRogge for awesome tutorials.




Comments:
Hi @NielsRogge, @sanchit-gandhi, dropout layers are being non-deterministic (with same random seed), without which I'm getting same outputs on my implementation and official codebase. 

seed code
```
seed_value = 42
torch.manual_seed(seed_value)
np.random.seed(seed_value)
random.seed(seed_value)
torch.use_deterministic_algorithms(True)`
```

Any suggestions on how to proceed further?

> use transformers.utils.set_seed if the old and new implementations are in the same framework.

[Hugging Face doc](https://huggingface.co/docs/transformers/en/add_new_model) says so.


Hi @ps4vs in PyTorch, you can just call `.eval()` on your model to turn off dropout.
max retries exceeded with url when trying to push to the hub
Author: avacaondata
Pull request: no

### System Info

the transformers library is irrelevant to this error, it has to do with the hub's s3 connections.

```
'(MaxRetryError("HTTPSConnectionPool(host='hf-hub-lfs-us-east-1.s3-accelerate.amazonaws.com', port=443): Max retries exceeded with url:
```

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

try to push a model to the hub and see the error: `'(MaxRetryError("HTTPSConnectionPool(host='hf-hub-lfs-us-east-1.s3-accelerate.amazonaws.com', port=443): Max retries exceeded with url:`

### Expected behavior

I'd expect that, though slow maybe (if the network is overloaded), models are pushed to the hub at some point. The cloud bill is getting much higher in the "pushing to the hub" part than in the training part (3k samples). 

Comments:
It still happens... I'm wasting tens of dollars in the cloud just trying to upload my model, any hints? I've tried using the `huggingface-cli` tool as well, with the same results.

cc @philschmid and @Wauplin you might have already seen this? 

Hi @avacaondata, I'm sorry you're facing those issues. These errors can happen when trying to upload files from behind a proxy. Could it be the case here as well? A good way to fix this is to set proxy config as environment variable (see [here](https://github.com/huggingface/transformers/issues/17611#issuecomment-1537709564)). A more dirty way of fixing it is to deactivate certificates (see [here](https://github.com/huggingface/transformers/issues/17611#issuecomment-1323272726)). Hope this can help you :) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`CI update`] Try to use dockers and no cache
Author: ArthurZucker
Pull request: yes

# What does this PR do?

So we have: 
### 1. `setup_and_quality` 83% faster on average, instead of a total of 11min it's down to 2min36.
I think we can do even better with `check repo consistency` by reducing the size of the docker used! (purging nvidia-*, and else not needed for checks). Breakdown:
- 2x on check circle ci user (from 14sec -> 7s)
- 10x on check code quality (from 6min -> 30s)
- 2x on check repo consistency   (from 1min -> 30s)
- 2x on test fetching (from 1min -> 30s)

### 2. `run_tests` from 17min on average to 6min
<img width="1059" alt="image" src="https://github.com/huggingface/transformers/assets/48595927/5f6899e7-58b1-4424-a9dc-c64d47d16dab">

before vs after:
![image](https://github.com/huggingface/transformers/assets/48595927/c665974a-15f5-473c-b0ff-8d9605060a2b)




Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_29202). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I mostly need the approval of the security team at this point...

<img width="1059" alt="image" src="https://github.com/huggingface/transformers/assets/48595927/10b7e0b0-c7a7-452e-94e8-d48aa078e3b1">



Wow, all ✅ ! I ❤️ seeing this. Will take a look this afternoon.
(But you can already remove `Draft` from the title 😉 )

(please wait for Yih-Dar's review before merging)
[i18n-<languageCode>] Translating docs to <languageName>
Author: Ggjkfkg
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu and @MKhalusova for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

<!--
Keep on adding more as you go 🔥
-->

Comments:
Hey @Ggjkfkg, I'm uncertain what language you'd like to translate here, so I'm closing the issue.
Quantization / HQQ: Fix HQQ tests on our runner
Author: younesbelkada
Pull request: yes

# What does this PR do?

Link to failing job: https://github.com/huggingface/transformers/actions/runs/8955054995

Fixes the current failing CI for HQQ, using bf16 fails on our T4 runners since SDPA + bf16 is not supported and we simply put `cache_dir` as an optional argument

cc @SunMarc @amyeroberts 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_30668). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

#30669 will adress the failing tests