noamaanMulla-03/transformers-issues huggingface dataset:
[`LlamaConfig`] Nit: pad token should be None by default
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Does not set the padding token as `0` is for the `unk_token`. Though Llama has bytefallback support, meaning most of the time the unkown token will not be form the input, it can generate the `<unk>` token (probably very rarely). The pad token is used in the embedding and the unk embedding is not None.

This should warn user that are doing `SequenceClassification` that the padding token is not set.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
🌐 [i18n-KO] Translated `add_tensorflow_model.md` to Korean
Author: keonju2
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `add_tensorflow_model.md` file of the documentation to Korean 😄 
Thank you in advance for your review!

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR?  -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- May you please review this PR? -->

Comments:
Please stop opening and closing the same PR like this.
Llama weights are in `bfloat16` but loaded as `float32`
Author: ayaka14732
Pull request: no

### System Info

- `transformers` version: 4.32.0.dev0
- Platform: Linux-5.8.0-1035-gcp-x86_64-with-glibc2.31
- Python version: 3.11.4
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
- Accelerate version: 0.21.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.0.dev20230719+cpu (False)
- Tensorflow version (GPU?): 2.14.0-dev20230719 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.13
- JaxLib version: 0.4.13
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
model = LlamaForCausalLM.from_pretrained(path)
print(model.model.norm.weight.detach().dtype)
```

This prints `torch.float32`.

### Expected behavior

This result is `torch.float32`, which indicates that the model is loaded in `float32` format. However, the result should be `torch.bfloat16` because it is the default for Llama models. The model weights are in `bfloat16` format.

Comments:
Hey! 👋🏻 
Without the path to the checkpoint that you are using, it's gonna be a bit hard for me to debug. The `dtype` can be affectect by the `model.config.torch_dtype` which is stored online. 

It is expected behavior: https://huggingface.co/docs/transformers/v4.31.0/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.torch_dtype. You can try to pass `torch_dtype="auto"` to `from_pretrained` and if the config.json is well written, it should load in the dtype specified in the config.

In general you should specify the dtype in which you want the model with `torch_dtype` (using auto is not recommended as configs are not always well written). In PyTorch models are always loaded in float32 by default (even if the state dict has another dtype), this is not something specific to Transformers.
Remove tokenizers from the doc table
Author: sgugger
Pull request: yes

# What does this PR do?

As discussed offline with @LysandreJik and @stas00 it doesn't really make sense to have the info on tokenizers in this table in the index, which:
- makes it look like something is missing for vision models or speech models
- is not accurate when a model uses another's tokenizer model.

This PR removes it and only leaves the frameworks supported by each model.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix type annotation for deepspeed training arg
Author: sgugger
Pull request: yes

# What does this PR do?

#24550 wanted to put a more exact type annotation for the deepspeed arg, which then makes that training arg fail in CLI commands. This PR reverts that part and adds a comment so we do not break this again.

Fixes #24974 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
facebook\opt layer norm
Author: ghost
Pull request: no

### System Info

transformers version 4.28.1.

I notice that in the facebook\optX models the LayerNorm weight is equal to 1 in all layers, means no parameter changed.

I checked the sizes 125m, 1.3b, 2.7b, 6.7b, 13b




### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

from transformers import  OPTModel
import torch

model = OPTModel.from_pretrained("facebook/opt-13b")
for m in model.modules():
      if isinstance(m,torch.nn.LayerNorm):
            (m.weight == 1).all()



### Expected behavior

I get: (Expected to get different values)

tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)
tensor(True)

Comments:
Hi @CompressTeam, thanks for raising this issue!

I believe this behaviour is likely coming from the fact that the layer norm layers are instantiated with `elementwise_affine=True` e.g. [here](https://github.com/huggingface/transformers/blob/fdd78d91532dffc4b2493d3b9bd9e19aaf78fe6b/src/transformers/models/opt/modeling_opt.py#L292) (as default [config value is `True`](https://github.com/huggingface/transformers/blob/fdd78d91532dffc4b2493d3b9bd9e19aaf78fe6b/src/transformers/models/opt/configuration_opt.py#L120)). This instantiates the layer with [all weight values as 1, and biases as 0](https://pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html). 

Playing quickly with the snippet provided, I can see that the biases are all different values, so it would seem that either only the biases were updated when training the model, there's been a error in weight conversion or an issue with weight saving. 

I'll hand over to @younesbelkada who added the model as is most familiar with layer norm related logic like `config._remove_final_layer_norm`




Hi @CompressTeam 

I think that this is expected, see this interesting thread from the authors: https://github.com/huggingface/transformers/issues/17653 and in particular these 2 messages: https://github.com/huggingface/transformers/issues/17653#issuecomment-1163065167 / https://github.com/huggingface/transformers/issues/17653#issuecomment-1163293340 from what I have understood the models somehow learned to get a layer norm of 1 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Error in finetuning starcoder with 8 GPU 24GB Memory 
Author: 22Mukesh22
Pull request: no

### System Info

I am trying finetuning starcoder , with 8 GPU P40 (each 24GB Memory) , I am using
 "https://github.com/Xirider/finetune-gpt2xl " and also referring the "https://github.com/bigcode-project/starcoder"
 Facing error with both. 
 
 The error says :
 raise ValueError( ValueError: You can't train a model that has been loaded in 8-bit precision on multiple devices in any distributed mode. In order to use 8-bit models that have been loaded across multiple GPUs the solution is to use Naive Pipeline Parallelism. Therefore you should not specify that you are under any distributed regime in your accelerate config.

@younesbelkada , please help on the same 
Thanks!!!

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

/group-volume/orc_srib/mukesh.sm/STARCODER/finetune-gpt2xl/deep/lib/python3.8/site-packages/accelerate/accelerator.py"
python -m torch.distributed.launch \
  --nproc_per_node 8 finetune.py \
  --model_path="bigcode/starcoder"\
  --dataset_name="HuggingFaceH4/CodeAlpaca_20K"\
  --split="train"\
  --size_valid_set 10000\
  --streaming \
  --seq_length 2048\
  --max_steps 1000\
  --batch_size 4\
  --input_column_name="prompt"\
  --output_column_name="completion"\ 
  --gradient_accumulation_steps 16\
  --learning_rate 1e-4\
  --lr_scheduler_type="cosine"\
  --num_warmup_steps 100\
  --weight_decay 0.05\
  --output_dir="./checkpoints" \


### Expected behavior

I am not able to understand why its not getting finetuned either 8 bit or 4 bit .
Because full precision if i try gives out of memory error . 
Please help with the right steps to finetune the starcoder . 

Comments:
Hi @22Mukesh22 
Thanks for the issue, Per my understanding you want to use NPP(Naive Pipeline Parallelism) 
For reference check: https://github.com/huggingface/accelerate/issues/1515#issuecomment-1584515731 and the entire thread
In this case you should run your script in a non-distributed mode. Please run your script with just `python  finetune.py xxxx` and let us know how it goes

But , with single GPU , it will take lot and lot of time to finish the training , I want to make sure it uses all my GPU , but its not happening .
I will update the same to you , for single GPU 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Use `accelerate` with transformers==4.26.1
Author: bhavitvyamalik
Pull request: no

### Feature request

How can we use `accelerate` features with an older version of `transformers`? I'm asking this because I've to use `adapter-transformers` and that is based on `transformers v4.26.1`


### Motivation

I used it latest `transformers` version and found it really helpful for parallelising the training 

### Your contribution

It's more of an optimization thing for an older version, I'm not sure how helpful will it be for other people or if any PR will be required for this.

Tagging @muellerzr here as he has worked on this perviously. Thank you!

Comments:
You'd need to recreate everything that has been performed throughout this integration, which is quite a lot. Distributed training still works in the Trainer natively, we just replaced its guts with Accelerate. The only real main difference is the DataLoader setup is a bit different in how it handles the batches. However in terms of usage nothing truly has changed. So they should update their version to v4.30.1 if you'd like this.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix graph break for Segformer
Author: amyeroberts
Pull request: yes

# What does this PR do?

Small change in the forward pass of Segformer which reduces the number of graph breaks in the compiled model from 5 to 0. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Closing as this is resolved on torch nightly
🌐 [i18n-KO] Updated Korean `serialization.md`
Author: wonhyeongseo
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.md` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Updated the `serialization.md` file for the Korean documentation.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
=Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

지속적인 번역 수정 작업 매우 멋집니다! 덕분에 ONNX 문서를 꼼꼼히 읽을 수 있었습니다. 수정할 부분은 없어보입니다!

@sgugger, @ArthurZucker, @eunseojo May you please review this PR?
The difference in length is due to an overhaul in the English document. I will try to use the same PR steps as https://github.com/huggingface/transformers/issues/20179#issuecomment-1528191933 for easier review next time.

Thank you so much for your support. I hope you have a great weekend! ❤️
Why the pre-trained models will be downloaded each times?
Author: xf05888
Pull request: no

I often use pre-trained models, but when I want to import it (after restart). 
But each time, I have to download again. So is there a function that can automatically save the model you downloaded in a directory? In Gluon-NLP, it will save the model you used in `.mxnet` in your home directory.

Comments:
They should definitely be cached locally.

Do you have a sample code showing the behaviour of re-downloading every time?

Yes, here is a example (not happen every time but after system restart):
```
import torch
from transformers import AlbertTokenizer
albert_tokenizer = AlbertTokenizer.from_pretrained("albert-xxlarge-v2")
```
It will show this progress bar each time I run this after shutdown or reboot:
**`Downloading: 100%|██████████████████████████████████████| 760k/760k [00:01<00:00, 556kB/s]`**

And I search the whole file system, no file match `albert` (even if I didn't restart the system).
So which directory should the model be under normal circumstances?

The models are by default cached to a hidden directory.

@AdityaSoni19031997 
Yes, I searched all the hidden directory as well and there's no eligible model file.

Even if there is, it will disappear after restart.

And what is the  name of the hidden directory?

@julien-c I found that it did cache in the local, but why it will disappear after **reboot**?
![Screenshot from 2020-03-17 18-01-14](https://user-images.githubusercontent.com/33285394/76915156-a7934700-6879-11ea-92e8-2f7b6199e52f.png)


Is their now a solution for that? In my case it tries to download the LLAMA-2 Model with almost 10 GB each time.
🌐 [i18n-KO] Translated`tasks/document_question_answering.md` to Korean
Author: jungnerd
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `tasks/document_question_answering.md` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
Team PseudoLab, may you please review this PR? 
@0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

May you please review this PR? 🤗
@sgugger, @ArthurZucker, @eunseojo
🌐 [i18n-KO] Translated `documnet question answering.mdx` to Korean
Author: jungnerd
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `document_question_answering.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of #20179 
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [ ] Check for missing / redundant translations (번역 누락/중복 검사)
- [ ] Grammar Check (맞춤법 검사)
- [ ] Review or Add new terms to glossary (용어 확인 및 추가)
- [ ] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [ ] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->

Comments:
Closing in favor of #24588