noamaanMulla-03/transformers-issues huggingface dataset:
A named Peft Model doesn't work with resume_from_checkpoint=True
Author: chenbin11200
Pull request: no

### System Info

transformers==4.36.2
peft==0.5.0


### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Hi @muellerzr and @pacman100,

It seem like if I try to resume a lora training by using 
```
trainer.train(resume_from_checkpoint=True)
```
it fails with the following error:
```
ValueError: Can't find a valid checkpoint at /my_output_dir/checkpoint-300
```
By checking the code, I figured out the resuming proccess is stopped  by the following assert in `Trainer._load_from_checkpoint` 
```
if not (
            any(
                os.path.isfile(f)
                for f in [
                    weights_file,
                    safe_weights_file,
                    weights_index_file,
                    safe_weights_index_file,
                    adapter_weights_file,
                    adapter_safe_weights_file,
                ]
            )
            or is_fsdp_ckpt
        ):
            raise ValueError(f"Can't find a valid checkpoint at {resume_from_checkpoint}")
```
Since I initialize the PeftModel with a adapter name, which is used to manage my adapters.
```
peft_model = get_peft_model(model=base_model,
                                       peft_config=peft_config,
                                       adapter_name='my_lora_model_name',
```
In this case, the `adapter_config.json` and `adapter_model.bin` files will be saved in `/my_output_dir/checkpoint-300/my_lora_model_name` instead of `/my_output_dir/checkpoint-300` directly. That's why the ValueError is raised.


I am not sure whether it is a known issue, and the proper way to fix it? Or I have to write my own PeftTrainer to handle this issue?

Thank you in advance for your support.

Best regards.

### Expected behavior

Resume the training with a named peft model is supported.

Comments:
A PR is attached, if it is necessary: https://github.com/huggingface/transformers/pull/28547

Also cc @younesbelkada for PEFT

Thanks @amyeroberts @chenbin11200 - the PR makes sense to me ! I left a single comment 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[i18n-en] Translating docs to Japanese
Author: rajveer43
Pull request: no
Labels: name: WIP

Hi!

Let's bring the documentation to all the Japanese-speaking community 🌐 

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `ja` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `ja/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @stevhliu and @MKhalusova for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).



## Model doc section
- [ ] [deit.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/deit.md) #28302
- [ ] [deplot.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/deplot.md) #28302
- [ ] [deta.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/deta.md) #28302
- [ ] [detr.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/detr.md) #28302
- [ ] [dialogpt.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/dialogpt.md)
- [ ] [dinat.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/dinat.md)
- [ ] [dinov2.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/dinov2.md)#29272
- [ ] [distilbert.md](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_doc/distilbert.md)

Keep on adding more as you go 🔥



Comments:
@stevhliu would you add the more models here in the list so other can look at this issue and collaborate?

@stevhliu, @ArthurZucker  would you add the more models here in the list so other can look at this issue and collaborate?
Fix building alibi tensor when num_heads is not a power of 2
Author: abuelnasr0
Pull request: yes

# Fix building alibi tensor when `n_heads` is not a power of 2

when the `n_heads` of MPT model is not a power of 2 number (ex: 6), the function that build the alibi tensor will return with an error, you can check that by running the extra test case that I have added. This PR fixes that issue. 

## Before submitting
- [x] Did you write any new necessary tests?


@ArthurZucker and @younesbelkada



Comments:
Cohere embed - Sagemaker deploy - Should have a `model_type` key in its config.json
Author: pthd
Pull request: no

### System Info



- `transformers` version: 4.33.2
- Platform: macOS-14.2.1-arm64-arm-64bit
- Python version: 3.9.16
- Huggingface_hub version: 0.18.0
- Safetensors version: 0.4.0
- Accelerate version: 0.23.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.1.0 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed




### Who can help?

@philschmid 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

deployed according to 

https://huggingface.co/Cohere/Cohere-embed-multilingual-v3.0?sagemaker_deploy=true



W-Cohere__Cohere-embed-mult-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - ValueError: Unrecognized model in /.sagemaker/mms/models/Cohere__Cohere-embed-multilingual-v3.0. Should have a `model_type` key in its config.json, or contain one of the following strings in its name:





### Expected behavior

deployment succeeds but invocation raises error

```
W-Cohere__Cohere-embed-mult-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - ValueError: Unrecognized model in /.sagemaker/mms/models/Cohere__Cohere-embed-multilingual-v3.0. Should have a `model_type` key in its config.json, or contain one of the following strings in its name:
```

Comments:
Hi @pthd, thanks for raising an issue! 

I'd suggest opening a discussion on the [model's hub page](https://huggingface.co/Cohere/Cohere-embed-multilingual-v3.0). As it looks like it might be related to the model itself and the owners of the repo will be able to address that. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fine tuning whisper and whisper lora with prompts
Author: kenfus
Pull request: no

### Feature request

Currently, I am successfully able to fine-tune whisper and whisper Lora with timestamps, thank you for that!

Now, I would like to fine tune whisper with prompts, as Openai has trained it. There is not too much documentation, so maybe it's already in? Currently, my dataset has the following columns: `input_features`, `labels` and `prompt_ids`. The labels do currently not contain the `prompt_ids`. So my first question:

- If I add `prompt_ids` to the `labels` at the beginning, is it already correct? Will the huggingface library automatically cut the labels at the correct point and pass them to the model to start the decoding? I did not understand where this in the code exactly happens. 
- If not, where would it be best to add? I think either it should happen automatically from the `label` or we could make the trainer use `prompt_ids` automatically, if available. 

### Motivation

Overall, it was a bit unclear on how to finetune whisper with timestamps and prompts. Maybe it's already there, maybe not. In addition, the code was a bit hard to understand in the HU library.

### Your contribution

Absolutely! If I get some guidance, where to look and what to change, I am happy to do so. I am happy to contribute to HU, which helped me a lot in my work.

Comments:
Hi @kenfus, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

cc @sanchit-gandhi @ylacombe 

Hey, thank you!

I have added a question to this post: https://discuss.huggingface.co/t/finetuning-whisper-with-prompts/43053/3

However, even when looking at the code, I don't quite understand how to pass the prompt to fine tune the model, so I think it's not implemented yet. With some guidance, I am very happy to implement it myself!

Hi @kenfus, 

I'll let @sanchit-gandhi and @ylacombe answer about fine-tuning. For reference on the intended API, the PR to add prompts is #22395. There is also another open PR looking to add prompts to the pipeline #28556. Although these are not specific to the training regime, they may provide more context on how to pass prompts to the model. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
The generation speed on NPU is too slow
Author: hhllxx1121
Pull request: no

The generation speed on NPU device is too slow，The first conversation takes about 5 minutes, and it may be faster next. May I ask if there is any error? Below is my code demo

```python
import torch
import torch_npu
from transformers import LlamaForCausalLM, LlamaTokenizer, TextStreamer



tokenizer = LlamaTokenizer.from_pretrained(
    "",
    device_map="npu:2"
)


llama_model = LlamaForCausalLM.from_pretrained(
    "",
    device_map="npu:2"
)
streamer = TextStreamer(tokenizer)


while True:
    ins = input("user: ")
    res = tokenizer.encode(ins, return_tensors="pt").to("npu:2")

    outputs = llama_model.generate(
        inputs=res,
        streamer=streamer,
        max_new_tokens=100,
    )
```

Comments:
cc @gante 

Hi @hhllxx1121 👋 

See this issue and its comments: https://github.com/huggingface/transformers/issues/28075

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
PermissionError occurs when calling Trainer.trainer using transformers
Author: Mickls
Pull request: no

### System Info

- `transformers` version: 4.37.1
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.10.13
- Huggingface_hub version: 0.20.3
- Safetensors version: 0.4.2
- Accelerate version: 0.26.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@muellerzr @pacman100

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The error code block comes from the official example https://huggingface.co/docs/transformers/tasks/sequence_classification
The following is the specific code
```python
training_args = TrainingArguments(
    output_dir="my_awesome_model",
    learning_rate=2e-5,
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    num_train_epochs=2,
    weight_decay=0.01,
    evaluation_strategy="epoch",
    save_strategy="epoch",
    load_best_model_at_end=True,
    push_to_hub=False,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_imdb["train"],
    eval_dataset=tokenized_imdb["test"],
    tokenizer=tokenizer,
    data_collator=data_collator,
    compute_metrics=compute_metrics,
)

trainer.train()
```

### Expected behavior

In trainer.py:2418 line `fd = os.open(output_dir, os.O_RDONLY)`, if the windows system tries to open a folder, a PermissionError exception will be triggered, so this piece of code will cause the train function to save the trained model to be interrupted.If possible, I hope you can be compatible with windows platform

Comments:
Hi @Mickls, thanks for raising this issue! 

A fix was merged in #28637. Could you try installing transformers from source and see if this resolves the issue for you? 

`pip install git+https://github.com/huggingface/transformers`

> Hi @Mickls, thanks for raising this issue!
> 
> A fix was merged in #28637. Could you try installing transformers from source and see if this resolves the issue for you?
> 
> `pip install git+https://github.com/huggingface/transformers`

Yes it works fine now, thanks

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Difficulty in adding custom model
Author: Frankenstein-hsa
Pull request: no

### Feature request

Hi 
Hope all the team members of hugging face are well 

I am a student and currently doing work on nlp projects , although most of my journey was successful because well documented information for starters especially example notebooks but what part is confusing and difficult is to upload and create a custom model from scratch i have and several other users have difficulty in it could you please make a notebook for step by step guidence to help me and other researchers to focus on thier projects not on these lenghty procedures 

### Motivation

Difficulty in portting custom model

### Your contribution

Student 

Comments:
Hi @El-chapo-007, thanks for opening this issue! 

Glad to hear that your journey has been mostly successful 🤗 

Have you seen our documentation page about adding custom models? This should contain all the info and example code needed to get started: https://huggingface.co/docs/transformers/custom_models

Let us know if anything does work! 

could you please add a Jupiter notebook template, as it would be alot more helpful as most of the other parts the hugging face team has put as a tutorial to better familirize with enviornment .....
And also my custom architecture  projects feed forward layer to the multiple of d_model,
as other models project it back to same dimension it just up_project and i have tested several other models but this model yeilds order of magnitude more performance with respect to number of parameters,hence i can deploy it on edge devices ..

But the issue as far as i know i don't have to use lm head liner layer but hugging face library automatically put we call a certain class for example AutoModelForCausalLM....

Since my model dimensions are same but in final liner layer how can I project it to multiple of d_model to final linear layer ...


also there is a bit confusion on https://huggingface.co/docs/transformers/custom_models   
vs  https://huggingface.co/docs/transformers/add_new_model

Hi @El-chapo-007, 

We can definitely think about adding a jupyter notebook. In the meantime, you should be able to run the code snippets in the documentation in cells in your own notebook.

I'm not sure I understand your question about modifying the models. However, this is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

Regarding the documentation pages: 
* https://huggingface.co/docs/transformers/add_new_model outlines how to add a model into the transformers repo
* https://huggingface.co/docs/transformers/custom_models outlines how to add a model on the hub


thanks alot

@El-chapo-007 you might consider checking this blogpost too 
https://huggingface.co/blog/not-lain/custom-architectures-with-huggingface
OSError: You are trying to access a gated repo.
Author: zhouhang-jpg
Pull request: no
Labels: name: New model

### Model description

I tried to run the model on Colab and successfully logged in using huggingface cli login, but it still reported an error
<img width="534" alt="联想截图_20240226102113" src="https://github.com/huggingface/transformers/assets/83337970/4d3d95a0-98c8-4451-b696-6a70f0d6c7ce">
<img width="1202" alt="联想截图_20240226102157" src="https://github.com/huggingface/transformers/assets/83337970/565471b5-0261-4420-9480-ec99d367b239">


### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Trainer is DP? support DDP？
Author: ciaoyizhen
Pull request: no

### Feature request

Is the trainer DDP or DP? If it is DDP, why do I train with multiple graphics cards, and the graphics card memory consumed on cuda-0 is much larger than other graphics cards. Or is it that when I increase per_device_train_batch_size, the cuda-0 card will exceed the graphics card memory, and then it will cut the model parameters to other cards by itself? Or do I need to set any parameters? Just give an example. I ask ChatGPT. He answer me that Trainer is DP.

### Motivation

DDP is more useful than DP.

### Your contribution

If supported. Could you tell me how to use DDP?

Comments:
Can it be DDP by adding fsdp="shard_grap_op" of TrainingArguments and fsdp=""full_shard" be DP?

Hi @ciaoyizhen, thanks for raising an issue! 

This is a question best placed in our [forums](https://discuss.huggingface.co/). We try to reserve the github issues for feature requests and bug reports.

We suggest searching in the forums first to see if any related questions have been asked e.g. https://discuss.huggingface.co/t/which-data-parallel-does-trainer-use-dp-or-ddp/16021/2. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
This model has one file that has been marked as unsafe. [training_args.bin]
Author: rizwan-ai
Pull request: no

### System Info

**Framework versions**

- Transformers 4.36.2
- Pytorch 2.1.2+cpu
- Datasets 2.16.1
- Tokenizers 0.15.0


This model has one file that has been marked as unsafe.
[training_args.bin](https://huggingface.co/rizwan-ai/distilbert-base-uncased-finetuned-emotion/blob/main/training_args.bin)

### Git LFS Details

* **SHA256:** d672df2806e4b013fbfdf9d995526b2c4e4a7d56a8b84b77b1d6213241ea11f0
* **Pointer size:** 129 Bytes
* **Size of remote file:** 4.73 kB

#### Detected Pickle imports (9)

* "transformers.training_args.TrainingArguments",
* "transformers.training_args.OptimizerNames",
* "transformers.trainer_utils.SchedulerType",
* "accelerate.state.PartialState",
* "torch.device",
* "transformers.trainer_utils.HubStrategy",
* "accelerate.utils.dataclasses.DistributedType",
* "__builtin__.getattr",
* "transformers.trainer_utils.IntervalStrategy"

@ArthurZucker @younesbelkada @pc

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://huggingface.co/rizwan-ai/distilbert-base-uncased-finetuned-emotion

### Expected behavior

How to fix it?

Comments:
Hi @rizwan-ai 
If you uploaded the files by yourself, in theory it should be safe, however we advise you to double check things before running anything. Please refer to this documentation page: https://huggingface.co/docs/hub/security-pickle for more details

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Feature Request: Expose an Args to Set Prefetch Factor in Trainer
Author: uygnef
Pull request: no

### Feature request

Currently, the trainer does not allow users to set the prefetch factor as an argument from the training script. 

### Motivation

This can be a limitation when training large models, especially when the data is being fetched from a remote server, as the loading speed can become a bottleneck due to slow data download for the next partition.

### Your contribution

I can commit a PR

Comments:
Hi @uygnef, thanks for raising an issue!

There is a related PR open: #28498 cc @muellerzr @pacman100 

> Hi @uygnef, thanks for raising an issue!
> 
> There is a related PR open: #28498 cc @muellerzr @pacman100

oh, good job!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.