noamaanMulla-03/transformers-issues huggingface dataset:
Extend Trainer to enable Ascend NPU to use the fused Adamw optimizer when training
Author: ji-huazhong
Pull request: yes

# What does this PR do?

This PR intends to extend Transformers Trainer Class with NpuFusedAdamw optimizer for model training when using Ascend NPU.

Verified with text-classification task:
```bash
export TASK_NAME=sst2

python run_glue.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
  --do_train \
  --do_eval \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --optim adamw_torch_npu_fused \
  --output_dir ./output
```
The result of `train_samples_per_second` is as follows

| Device            | `adamw_torch`(default) | `adamw_torch_npu_fused`  |
| ----------------- | --------------------------------- | ------------------------ |
| Ascend 910B     | 96.586                              | 149.876                   |



## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerz and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->
cc @pacman100 and @muellerzr 

Comments:
Rebase my commits to master HEAD to avoid too many merges back from the master.

BTW, Is this PR still under review? Any review suggestions? Please let me know if there is anything else that needs to be done.
cc @muellerzr

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26194). All of your documentation changes will be reflected on that endpoint.
add bf16 mixed precision support for NPU
Author: ji-huazhong
Pull request: yes

# What does this PR do?
Accelerate already provides NPU support for mixed precision training using bf16. See https://github.com/huggingface/accelerate/pull/1949. This PR makes it available on Transformers 🤗

Verified on text classification example using the following script:
```
export TASK_NAME=mrpc

python run_glue.py \
  --model_name_or_path bert-base-cased \
  --task_name $TASK_NAME \
  --do_train \
  --do_eval \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 2e-5 \
  --num_train_epochs 3 \
  --bf16 \
  --output_dir ./output
```


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. 

@pacman100 and @muellerzr  Good day:-). Could you please take a look at this PR for me? Thanks.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26163). All of your documentation changes will be reflected on that endpoint.

@muellerzr sorry for bothering you. This PR is ready to be merged :D.
[docstring] Fix docstring for GPT2Config
Author: KMJ-007
Pull request: yes

# What does this PR do?
fixed doc string for GPT2Config

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
  @ydshieh


Comments:
@ydshieh all checks have passed!

can you review this PR

thankyou

closing this, this PR is already fixing this!

https://github.com/huggingface/transformers/pull/26642
Fix issue of canine forward requiring input_ids anyway
Author: marcmk6
Pull request: yes

The current `forward` requires (the shape of) `input_ids` for deriving other variables whenever `input_ids` or `inputs_embeds` is provided. Change this to use the given one instead of `input_ids` all the time.

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes  #26288 

## Who can review?

@ArthurZucker and @younesbelkada

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26290). All of your documentation changes will be reflected on that endpoint.
Inject the optimum.bettertransformer to trainer to accelerate training
Author: DongHande
Pull request: no

### Feature request

Inject the optimum (https://huggingface.co/docs/optimum/main/en/bettertransformer/tutorials/convert#training-compatibility) to the trainer API (https://github.com/huggingface/transformers/blob/main/src/transformers/trainer.py). 

### Motivation

The flash attention can accelerate the training speed and save memory a lot. And the optimum library implements the flash attention with a concise and elegant way which is quite easy to use. 

However, the trainer API does not support optimum up to now. If the trainer can support optimum, it will be quite convenient to train the LLMs. 

### Your contribution

Submit issues related to bugs or desired new features.

Comments:
cc @younesbelkada @fxmarty 

Hi @DongHande 
For now we leave it to users responsibility to first convert the model into `BetterTransformer` format and pass it to Trainer. Please have a look at this section of TRL documentation: https://huggingface.co/docs/trl/main/en/sft_trainer#using-flash-attention-and-flash-attention-2 that is also applicable for HF `Trainer` to enable Flash-Attention 1 and 2. 

The documentation your mentioned is clear. I learn to use flash attention from this documentation, and it is easy to use. 

Thanks for your reply.
cannot import name 'AutoModelForSequenceClassification' from 'transformers'
Author: nithinreddyy
Pull request: no

```
from transformers import pipeline
classifier = pipeline('sentiment-analysis') #This code will download the pipeline
classifier('We are very happy to show you the 🤗 Transformers library.')
classifier.save_pretrained('/some/directory') 
```

I'm trying to save the model and trying to perform the sentiment-analysis operation offline

```
from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

t = AutoTokenizer.from_pretrained('/some/directory')
m = AutoModelForSequenceClassification.from_pretrained('/some/directory')
c2 = pipeline(task = 'sentiment-analysis', model=m, tokenizer=t)

I'm facing import error for jupyter notebook as given below

`**cannot import name 'AutoModelForSequenceClassification' from 'transformers'**`
```

Comments:
Please tell us which version you are using and also check [your SO question](https://stackoverflow.com/questions/66906652/how-to-download-hugging-face-sentiment-analysis-pipeline-to-use-it-offline/66907181#comment118269245_66907181) again because it was revised.



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I am trying the below codes

```
from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
from scipy.special import softmax
```

however, getting the below errors in jupyter notebook

`---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[49], line 2
      1 from transformers import AutoTokenizer
----> 2 from transformers import AutoModelForSequenceClassification
      3 from scipy.special import softmax

ImportError: cannot import name 'AutoModelForSequenceClassification' from 'transformers' (C:\Users\Abhik\anaconda3\Lib\site-packages\transformers\__init__.py)`


are there any changes to it? I am using jupyter notebbok 7
[DOCSTRING] [Wip] Fix docstring DPT-Model
Author: AVAniketh0905
Pull request: yes

# What does this PR do?

Address #26638 

HELP NEEDED!

## Description
I have followed the guide to contributing for #26638 , but couldn't generate anything using

`python3 utils/check_docstrings.py --fix_and_overwrite`. 

The output,
```
2023-10-07 12:47:24.603397: E tensorflow/compiler/xla/stream_executor/cuda/cuda_dnn.cc:9342] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2023-10-07 12:47:24.603448: E tensorflow/compiler/xla/stream_executor/cuda/cuda_fft.cc:609] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2023-10-07 12:47:24.603485: E tensorflow/compiler/xla/stream_executor/cuda/cuda_blas.cc:1518] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2023-10-07 12:47:25.616995: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
```

A similar issue: [Tensorflow](https://github.com/tensorflow/tensorflow/issues/62002)

The docstring did not miss anything thus i have removed `DPTModel` from
 `utils/check_docstrings.py` > `OBJECTS_TO_IGNORE`.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@ydshieh 

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
[DINOv2] Convert more checkpoints
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR converts the DINOv2 checkpoints of image classification on ImageNet-1k ([source](https://github.com/facebookresearch/dinov2#pretrained-heads---image-classification)).

It also improves the doc test for `Dinov2ForImageClassification`.

Fixes #26167

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks !
Fixed malapropism error
Author: Zhreyu
Pull request: yes

# What does this PR do?

This PR fixes a minor typo in the code comment. It replaces the word "clone" with "copy" in the comment to improve clarity.

## Before submitting
- [X ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag members/contributors who may be interested in your PR.





Comments:
fix links in README.md for the GPT, GPT-2, and Llama2 Models
Author: dcarpintero
Pull request: yes

# What does this PR do?

Fix broken links as follows:

[-] https://blog.openai.com/language-unsupervised/
[+] https://openai.com/research/language-unsupervised/

[-] https://blog.openai.com/better-language-models/
[+] https://openai.com/research/better-language-models/

[-] https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/XXX
[+] https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/

Comments:
Avoid CI OOM
Author: ydshieh
Pull request: yes

# What does this PR do?

The torch and torch pipeline job again (almost) reach the limit of CircleCI runner's RAM (16 G).

torch pipeline job crashed [nightly run here](https://app.circleci.com/pipelines/github/huggingface/transformers/74561/workflows/5c4e2b07-0688-4d86-bac6-85250ebbd741/jobs/944985)

A screenshot (of [another run](https://app.circleci.com/pipelines/github/huggingface/transformers/74427/workflows/d5130b1e-0bec-4b41-867c-7dd61722434e/jobs/942978/resources))

<img width="1061" alt="Screenshot 2023-10-06 175412" src="https://github.com/huggingface/transformers/assets/2521628/2f9dc0b6-3a0d-46a6-88c6-bd1e787f06b6">

Let's use 6 workers for now: I will try to find time to investigate what happens recently.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[docstring] Fix docstring for DonutImageProcessor
Author: abzdel
Pull request: yes

fixed DonutImageProcessor docstring and removed from OBJECTS_TO_IGNORE in check_docstrings.py

Comments:
@ydshieh should be all set! Happy to make changes if needed

@ydshieh Just fixed this with my most recent commit. Just a heads up - I get the following when I run `make fixup`. I looked into fixing it but I wasn't quite able to - happy to keep investigating this if needed
<img width="544" alt="image" src="https://github.com/huggingface/transformers/assets/55398496/30946b54-5fc7-487a-9117-35333ed1134a">


It's probably a TF issue in your environment. Other people also reported this, but I have no clear idea what's going on here as it works for me as well as on CI runner. For this PR, as CI is green, good to merge.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_26641). All of your documentation changes will be reflected on that endpoint.
use_flash_attention_2=True for Llama2 breaks generation
Author: markovalexander
Pull request: no

### System Info

- `transformers` version: 4.34.0
- Platform: Linux-5.15.0-1042-oracle-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.2
- Accelerate version: 0.21.0
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no


### Who can help?

text models: @ArthurZucker and @younesbelkada


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Using flash attention 2 completely breaks generation. 

<img width="651" alt="image" src="https://github.com/huggingface/transformers/assets/22663468/22384c4a-0aa4-4a51-acc7-805379a2b72f">


### Expected behavior

Generations match

Comments:
It generates "nobody" on any prompt actually

Hi @markovalexander 
I did not managed to repro on an A100, the latest FA-2 release and HF transformers main branch:

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "meta-llama/Llama-2-7b-chat-hf"
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.float16, use_flash_attention_2=True, low_cpu_mem_usage=True).to(device)

inputs = tokenizer.encode("Hello how are you?", return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=4, do_sample=False)
print(tokenizer.decode(outputs[0]))
```

I get

```bash

 Hello how are you? I'm doing
```

What hardware are you using?

Hi @younesbelkada , thanks for looking into the issue. I am using A100, nvidia-smi first line is:

`| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |`

and 

```
❯ pip show flash_attn
Name: flash-attn
Version: 2.0.0.post1
```

Thanks, I am using

```bash
> pip show flash_attn
Name: flash-attn
Version: 2.3.1.post1
```

Can you try to use the latest FA package? that might be the culprit. `pip install -U flash-attn --no-build-isolationn`

Updating flash attention helped, thank you :) 

Awesome, thanks @markovalexander !
Training issue with the Transformer CAPTCHA recognition model: Unable to converge
Author: Arc-2023
Pull request: no

I have built a model from scratch, inspired by the Transformer model and related code (such as ViT), with the goal of recognizing CAPTCHAs. However, during training, I've encountered an issue with the Transformer model. After several batch iterations, I consistently observe that the highest probability value in the output probability matrix is `<EOS>`, and this problem persists even after prolonged training.

Here is an overview of my approach: I initially followed the ViT approach, where I divide input images into many small patches. Each patch is then linearly mapped to a fixed emb_d dimension. For the decoder, I map the CAPTCHA letters to the same fixed emb_d values (note: the vocabulary includes digits and letters [0-9a-zA-Z]). This way, I construct an input sequence for the encoder.

For the encoder, I use the image patches as input and pass them through multiple encoder blocks, each consisting of multi-head self-attention layers, layer normalization, residual connections, and linear layers. Finally, the encoder's output matches the input's shape, i.e., [batch len_batch emb_d], and this output serves as both the key and value matrices for the decoder.

For the decoder, I use the target sequence (with a shape of [batch len_batch emb_d] and the last token removed) as input and set the target sequence (with the first token removed) as the actual target. I then compute the cross-entropy loss between the output and the target.

![image](https://github.com/huggingface/transformers/assets/64178177/f3baa707-7b56-4335-8c66-197440e0c4bd)



The issues I've identified are as follows: In the screenshots, it's evident that after taking the argmax of the output probability matrix, it should yield the index of the predicted label (out), which ideally should match the target index (tgt). However, I've noticed that the output for 'out' consistently corresponds to index 1, indicating "`<EOS>`."


You can find the code for this top get the errors of structures in the following location：
> https://nbviewer.org/github/Arc-2023/IPYNB/blob/main/notebook59efbd9b73.ipynb



I have roughly verified the network structure and found no errors, but I remain uncertain. I hope someone can help me analyze this issue, and I would be extremely grateful for any assistance in resolving it.


  [1]: https://i.stack.imgur.com/UCtqy.png

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discuss.huggingface.co) instead?

Thanks! 

> Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests. Could you ask your question on the [forum](https://discuss.huggingface.co) instead?
> 
> Thanks!

Thank you for the suggestion, since it has been solved, thus I will close it, this is the jump link: 
> https://discuss.huggingface.co/t/training-issue-with-the-transformer-captcha-recognition-model-unable-to-converge/57334/3