noamaanMulla-03/transformers-issues huggingface dataset:
Running official `apply_chat_template` throws warnings
Author: Boltzmachine
Pull request: no

### System Info

- `transformers` version: 4.34.0
- Platform: Linux-5.10.0-26-cloud-amd64-x86_64-with-glibc2.31
- Python version: 3.9.18
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: 0.23.0
- Accelerate config:    not found
- PyTorch version (GPU?): 1.13.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@ArthurZucker 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the official scripts from https://huggingface.co/docs/transformers/chat_templating

```python
>> from transformers import AutoTokenizer
>> tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf")

>> chat = [
  {"role": "user", "content": "Hello, how are you?"},
  {"role": "assistant", "content": "I'm doing great. How can I help you today?"},
  {"role": "user", "content": "I'd like to show off how chat templating works!"},
]

>> tokenizer.use_default_system_prompt = False
>> tokenizer.apply_chat_template(chat, tokenize=False)
"

 Hello, how are you? 
 I'm doing great. How can I help you today? 


 I'd like to show off how chat templating works! 
"
```

This throws warnings on my end
```
>>> tokenizer.apply_chat_template(chat, tokenize=False)
Using sep_token, but it is not set yet.
Using pad_token, but it is not set yet.
Using cls_token, but it is not set yet.
Using mask_token, but it is not set yet.
```


### Expected behavior

No warnings

Comments:
Hey this is a tokenizer issue and will be fixed by #26570

Any local workaround I can make to avoid this message?

Yes you can initialize the tokenizer with `verbose = False`:
```python 
>> from transformers import AutoTokenizer
>> tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf", verbose = False)
```
Emergency PR to skip conversational tests to fix CI
Author: Rocketknight1
Pull request: yes

The CI is currently red because of issues with the conversational pipeline tests on several models. This is caused by my recent PR #26795. I'm not sure why the issue didn't appear in that PR - maybe the tests weren't triggered for some reason.

This emergency temporary PR skips the tests for now while I work on a proper solution!

Comments:
I'm seeing some issues with BlenderBot as well, so it's possible that not all tests are triggering in every CI run! I think to avoid blocking lots of other PRs, we should just disable the test until I can test everything and re-enable it.
[performance] module init w/ `from_pretrained` skip storage allocation
Author: stas00
Pull request: no
Labels: name: WIP

# 🚀 Feature request

pt-1.9.0 added `torch.nn.utils.skip_init()` which (1) skips the module init (2) doesn't allocate any memory
https://pytorch.org/tutorials/prototype/skip_param_init.html

note: `torch.nn.utils.skip_init()` itself will be in 1.9.1, but the rest of the code should be in 1.9.0 (update: as 1.9.1 isn't planned, probably `s/1.9.1/1.10/`)

We already implemented part 1 (skipping the custom init) in https://github.com/huggingface/transformers/pull/11471.

We could further speed up the start up time and reduce CPU memory usage, by not allocating any storage for module init since `load_state_dict` will already have allocated `state_dict` from the pretrained weights (and some sub-modules that don't have pre-trained weights - will have to go through normal init). See https://pytorch.org/tutorials/prototype/skip_param_init.html#implementation-details

another note: currently deepspeed needs to have the module storage pre-allocated for its `zero.Init` gather/scatter, but if the initial model's weights aren't allocated, then we can probably get rid of  `zero.Init` altogether https://github.com/huggingface/transformers/issues/12273

Comments:
If this issue hasn't already been resolved and a fix is relevant, can I have a try at it @stas00?

Thank you for offering to implement this, @tanaymeh 

I think this is no longer relevant, as recent pytorch versions added allocation on meta device, which does the same and should be used instead, so closing this.
Mark test_add_special_tokens as slow for Whisper
Author: Rocketknight1
Pull request: yes

test_add_special_tokens has been causing some CI timeouts, and the test seems to be very slow for Whisper when I test it locally. I'm marking the test as `@slow` so it stops causing CI issues, but still runs nightly.

cc @ydshieh 

Comments:
Closing this because it seems to be superseded by #26570
🌐 [i18n-ZH] Translated fast_tokenizers.md to Chinese
Author: yyLeaves
Pull request: yes

# What does this PR do?
Add zh (Chinese) translation for fast_tokenizers.md. #20095 
## Who can review?
Documentation: @stevhliu 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
Add default template warning
Author: Rocketknight1
Pull request: yes

Lots of user repos use common tokenizers like `LlamaTokenizer` even for non-LLaMA models. Unfortunately, these tokenizers come with default chat templates, which may result in user confusion if they use `apply_chat_template`. We want to avoid a scenario where users think they're getting the right template but actually aren't!

This PR raises a short warning the first time a `default_chat_template` is read, which happens when `apply_chat_template` or `ConversationalPipeline` is called for a model without a `chat_template` set. The warning tells the user what's happening, and suggests adding an explicit chat template.

Over time, the goal is to eventually deprecate and remove `default_chat_template`, because we want to avoid using class-level templates entirely, and move to explicit repo level `chat_template` attributes. This PR starts that process, while still retaining the feature for backward compatibility.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26637). All of your documentation changes will be reflected on that endpoint.

Hmmm I'm not sure this warning will be useful for the majority of users; wouldn't it be better as an INFO level? 

I think having class-level templates is better than not having anything, and IMO it would make sense to have a sensible default (original architecture training scheme) rather than forcing each checkpoint to have their own definition. Or do you expect every checkpoint to have a different template? 

Just from looking at models on the Hub, I think a lot of models use templates totally different from the default class one. For example, if you look at the top trending models on the Hub today, the top model is `Mistral-7B-instruct`, which uses `LlamaTokenizer` with a custom chat template, and the second place model is `Mistral-7B-OpenOrca`, which also uses `LlamaTokenizer` but with a ChatML template. Neither of these templates match the class-level template.

The problem right now is if a user uses `apply_chat_template` with a model like that which doesn't have a `chat_template` attribute, it will appear to work, but they'll silently get the wrong, class-level template instead! This is exactly what we want to avoid with chat templates.

I think raising a warning is the right approach - it won't break existing workflows, but it will flag a potential source of issues, and it might notify users and maintainers to add proper chat templates to the models they're using.

Ok, sounds good to me in that case. Thanks for the explanation!

Sure! Also @ArthurZucker to respond to your comments, I'm probably going to start with this, then maybe we can begin deprecating `default_class_template` over time, but I want to do it slowly because it breaks old workflows. 

The `FutureWarning` suggestion was good, though, I'll add that now!

`FutureWarning` added!

Update: After testing, I took `FutureWarning` out because it clashes with `warning_once`, and I don't want to spam these messages too much!
Refactor code part in documentation translated to japanese
Author: rajveer43
Pull request: yes

# What does this PR do?

this PR fixes some coding examples which was translated with the original content to japanese text. refactoring that thing in this PR.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #26895

@stevhliu 


Comments:
some code part in documentation was translated to japanese
Author: rajveer43
Pull request: no

### System Info

https://moon-ci-docs.huggingface.co/docs/transformers/pr_1/ja/preprocessing#pad

some code part in documentation was mistakenly translated to Japanese.

 

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

-

### Expected behavior

```
batch_sentences = [
    "でもセカンドブレックファーストはどうなるの？",
    "セカンドブレックファーストについては知らないと思う、ピップ。",
    "イレブンジーズはどうなの？",
]
encoded_input = tokenizer(batch_sentences, padding=True)
print(encoded_input)
```

should be

```python
batch_sentences = [
    "でもセカンドブレックファーストはどうなるの？", #english text
    "セカンドブレックファーストについては知らないと思う、ピップ。", #english text
    "イレブンジーズはどうなの？", #english text
]
encoded_input = tokenizer(batch_sentences, padding=True)
print(encoded_input)
```

Comments:
cc @stevhliu
[i18n-ZH] Translated fast_tokenizers.md to Chinese
Author: yyLeaves
Pull request: yes

# What does this PR do?
Add zh (Chinese) translation for fast_tokenizers.md. #20095 
## Who can review?
Documentation: @stevhliu 
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.



Comments:
The hidden states in LlamaFlashAttention2 are cast in fp16 unexpectedly
Author: hiyouga
Pull request: no

### System Info

- `transformers` version: 4.33.1
- Platform: Linux-5.4.0-147-generic-x86_64-with-glibc2.31
- Python version: 3.10.13
- Huggingface_hub version: 0.17.1
- Safetensors version: 0.3.3
- Accelerate version: 0.23.0
- PyTorch version (GPU?): 2.0.1+cu118 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: A100 40GB
- Using distributed or parallel set-up in script?: No

### Who can help?

@younesbelkada 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

As we discussed in this thread: https://github.com/huggingface/transformers/pull/25598#discussion_r1338877983

The hidden states may be cast in float16 even if we are using bf16 mixed precision training.

https://github.com/huggingface/transformers/blob/78dd1202823ca035b9609ddbcdaac2945a6530ff/src/transformers/models/llama/modeling_llama.py#L485-L487

It may be difficult to figure out the correct data type if the model is loaded in 4/8-bit mode.


### Expected behavior

The hidden states should be cast in Bfloat16 in bf16 training.


Comments:
Indeed we should not always cast if the dtype is float32

FYI @younesbelkada 

Thanks @hiyouga this makes sense

> Indeed we should not always cast if the dtype is float32

Flash Attention supports only fp16 / bf16 as input dtype so we should always cast to half precision if the input gets silently casted to full precision (e.g. layer norm in Llama)

I will work on it and let you know !
[`FA-2`] Final fix for FA2 dtype
Author: younesbelkada
Pull request: yes

# What does this PR do?

Replaces https://github.com/huggingface/transformers/pull/26560 
Fixes https://github.com/huggingface/transformers/issues/26451

Proposes a simpler fix for dealing with FA-2 + PEFT + quantization fine-tuning where users usually cast all other modules (e.g. LayerNorms) in fp32 for training stability. 

With https://github.com/huggingface/transformers/pull/26761 being introduced, it is now much simpler to retrieve model's original dtype, note also that `self.config._pre_quantization_dtype` remains the single source of truth as `to` is not supported for quantized models

cc @ArthurZucker @pacman100 

Added also a nice test

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26846). All of your documentation changes will be reflected on that endpoint.
[`FA-2`] Revert suggestion that broke FA2 fine-tuning with quantized models
Author: younesbelkada
Pull request: yes

# What does this PR do?

Reverts: https://github.com/huggingface/transformers/pull/26846#discussion_r1361616963

This leads to an error because `getattr` will try to evaluate the third argument. for quantized models `weight` attribute do not exist leading to an error making the fine-tuning not possible

To repro on a more local scope:

```python
>>> import torch
>>> t = torch.randn(1)
>>> getattr(t, 'device', t.non_existent_attribute)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Tensor' object has no attribute 'non_existent_attribute'
```

cc @ArthurZucker 


Comments:
[docstring] Fix bert japanese tokenizer
Author: przemL
Pull request: yes

# What does this PR do?

Fix docstring for bert japanese tokenizer.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ydshieh

Comments:
@ydshieh Could you please review the PR?
Thank you.
[docstring] Fix bit image processor
Author: przemL
Pull request: yes

# What does this PR do?
Fix docstring for bit image processor.

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ydshieh

Comments:
@ydshieh Could you please review the PR?
Thank you.

Hi @przemL I see you closed this (and another) PR. Any particular reason? Feel free to reopen is you still want to have your contribution being merged. Thank you.
Falcon tokenizer doesn't use eos_token.
Author: zhaowei-wang-nlp
Pull request: no

### System Info

transformers==4.34.0
python==3.9.17

### Who can help?

@ArthurZucker @youne

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
# Load model directly
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-7b")
token_ids = tokenizer("I eat an apple")["input_ids"]
print(tokenizer.convert_ids_to_tokens(token_ids))
token_ids = tokenizer("I eat an apple", "I eat an banana")["input_ids"]
print(tokenizer.convert_ids_to_tokens(token_ids))
```


### Expected behavior

Current output of "tokenizer("I eat an apple", "I eat an banana")":
<img width="843" alt="image" src="https://github.com/huggingface/transformers/assets/22047467/616a49f0-f0db-4432-8ed6-31447530e1e5">
There is not eos_token between two sentences.

The expected behavior should be a eos_token/[SEP] token between these two sentences.
bert-base-uncased's behavior:
<img width="1297" alt="image" src="https://github.com/huggingface/transformers/assets/22047467/cefd450c-b03e-46b4-82a8-c1a0b03853c1">




Comments:
Fix and re-enable ConversationalPipeline tests
Author: Rocketknight1
Pull request: yes

The bug didn't turn out to be too bad - some models just had very short `max_position_embeddings` in their test configs, which meant the conversation tests generated outputs that were too long. Limiting `max_new_tokens` seems to have fixed it, but I'm running other tests to be sure!

Comments:
Looks good - ready to merge after review!

@ArthurZucker I tried that, but sometimes the tests get very slow when `max_position_embeddings` is large! Using a small number like 10 or 20 keeps the test quick.
Add fuyu model
Author: molbap
Pull request: yes

# What does this PR do?

This PR adds the model Fuyu from Adept, a LMM processing image and text inputs.


## Before submitting

- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?





Comments:
edit: a previous version of this comment referred to a bug in loading the model--I uninstalled and reinstalled `transformers` and it appears to have resolved itself.

However, the processor does not seem to support batch_size > 1 inputs correctly, the code  appears to construct the tensors correctly, but then slices to take their first element only (except the patch indices) [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/fuyu/processing_fuyu.py#L545-L546). Is this intentional?

How much VRAM does Fuyu-8B need to run? 

Is it possible to have quantized versions running in C like the models TheBloke provides?

1. You need around 20GB for the bfloat16
2. You can already use the quantisation config with the integration in transformers see [the doc here](https://huggingface.co/docs/transformers/main_classes/quantization). 
3. This kind of question would be best asked directly on the model [on the hub](https://huggingface.co/adept/fuyu-8b/discussions)!
[i18n-<languageCode>] Translating docs to <languageName>
Author: Jiuri05
Pull request: no

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu and @MKhalusova for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [ ] [index.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.md) https://github.com/huggingface/transformers/pull/20180
- [ ] [quicktour.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.md) (waiting for initial PR to go through)
- [ ] [installation.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.md).

## Tutorial section
- [ ] [pipeline_tutorial.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.md)
- [ ]  [autoclass_tutorial.md](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.md)
- [ ]  [preprocessing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.md)
- [ ]  [training.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.md)
- [ ]  [accelerate.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.md)
- [ ]  [model_sharing.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.md)
- [ ]  [multilingual.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.md)

<!--
Keep on adding more as you go 🔥
-->

Comments: