noamaanMulla-03/transformers-issues huggingface dataset:
Huggingface Transformers fastTokenizer for DeBERTa v3
Author: bluesky333
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

I want to try NER for custom dataset using huggingface transformers DeBERTa v3 xsmall.
I need to use fastTokenizer, but it is not available.
I need to use fastTokenizer for the return_offsets_mapping function.

Do you have plans for releasing fastTokenizer for DeBERTa v3?

When I run the below code,
`train_encodings = tokenizer(train_texts, is_split_into_words=True, return_offsets_mapping=True, padding=True, truncation=True, max_length=MAX_LENGTH)`

I get the below is the error message:
`NotImplementedError: return_offset_mapping is not available when using Python tokenizers.To use this feature, change your tokenizer to one deriving from transformers.PreTrainedTokenizerFast.`


Comments:
Found #14712 for the same request

Let's centralize the discussion on #14712 if you don't mind :)

i have same problem
Add FlaxWhisperForAudioClassification model
Author: raghavanone
Pull request: yes

# What does this PR do?

Fix : #21779 

Please review and let me know changes @sanchit-gandhi 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_21894). All of your documentation changes will be reflected on that endpoint.

> Modelling code looks good @raghavanone! Nice one on getting this working so quickly 🙌 Do you want to have a go at adding the encoder-only tests? See the PyTorch WhisperForAudioClassficiation PR for details, think you can also add these quite quickly :)

I have added the Encoder tests, But some test are failing, The FlaxWhisperForAudioClassification class extends FlaxWhisperPreTrainedModel . Due to this inheritance, the call method expects decoder related params. 

Should the FlaxWhisperForAudioClassification not extend FlaxWhisperPreTrainedModel instead create a new pretrainedclass ? 

Hey @raghavanone! The PyTorch model has just been merged (https://github.com/huggingface/transformers/pull/21754), so you can rebase onto main to get the required config changes:
```
git fetch upstream
git rebase upstream/main
```
This will fix the failing Flax tests we're getting here: https://app.circleci.com/pipelines/github/huggingface/transformers/58972/workflows/2388bd70-553e-412f-9ee7-0599cace5639/jobs/719829

The only thing to make sure is that the first time you push after rebasing, you **force push** to origin:
```
git add .
git commit -m "Some new changes after rebase"
git push -f origin fix_issue_21779
```

You only have to force push once, the next time you can just regular push:
```
git add .
git commit -m "Some more changes"
git push -u origin fix_issue_21779
```

@sanchit-gandhi  There are 2 test failing here, I am unable to get the same failure locally in my machine. Any pointers on how to replicate failing test and fix it ? 

Hey @raghavanone! Would you mind going through the previous review comments and marking them as resolved where you've addressed them? I'll then get you a final review asap! Thanks!

Hey @raghavanone - I think the commit history has been corrupted for this PR? Gentle reminder that one must force push after rebasing: https://github.com/huggingface/transformers/pull/21894#issuecomment-1458359220 Think this is probably the culprit for the 250 extra commits!

In this instance, it's probably best to close this PR in favour of a new one that only contains the new changes you with to merge. Sorry about that!

Closing in favour of #22883
[docs] Text to speech task guide
Author: MKhalusova
Pull request: yes

This PR adds a multimodal task guide on fine-tuning SpeechT5 for text-to-speech. It's based on a wonderfully [detailed notebook](https://colab.research.google.com/drive/1i7I5pzBcU3WDFarDnzweIj4-sVVoIUFJ#scrollTo=uELTb9CcOaCp) by @hollance.
 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

PR with images: https://huggingface.co/datasets/huggingface/documentation-images/discussions/86
fix spelling error
Author: digger-yu
Pull request: yes



# What does this PR do?
fix spelling error
change referrred to referred


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Error in get embedding_size.
Author: nursery42
Pull request: no

### System Info

- transformers version: 4.28.1
- deepspeed 0.8.3

In [examples](https://github.com/huggingface/transformers/tree/main/examples)/[pytorch](https://github.com/huggingface/transformers/tree/main/examples/pytorch)/[language-modeling](https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling)/[run_clm.py](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm.py)

when I set deepspeed config, I get an embedding_size as 0.




### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```
embedding_size = model.get_input_embeddings().weight.shape[0]
if len(tokenizer) > embedding_size:
    model.resize_token_embeddings(len(tokenizer))
```

At this point, the value of embedding_size obtained is 0, which may cause an error in code. Maybe it's because using deepspeed config. 

```
{
  "fp16": {
    "enabled": "auto",
    "loss_scale": 0,
    "loss_scale_window": 1000,
    "initial_scale_power": 16,
    "hysteresis": 2,
    "min_loss_scale": 1
  },
  "optimizer": {
    "type": "AdamW",
    "params": {
      "lr": "auto",
      "betas": "auto",
      "eps": "auto",
      "weight_decay": "auto"
    }
  },
  "scheduler": {
    "type": "WarmupDecayLR",
    "params": {
      "warmup_min_lr": "auto",
      "warmup_max_lr": "auto",
      "warmup_num_steps": "auto",
      "total_num_steps": "auto"
    }
  },
  "zero_optimization": {
    "stage": 3,
    "overlap_comm": true,
    "contiguous_gradients": true,
    "sub_group_size": 1e9,
    "reduce_bucket_size": "auto",
    "stage3_prefetch_bucket_size": "auto",
    "stage3_param_persistence_threshold": "auto",
    "stage3_max_live_parameters": 1e9,
    "stage3_max_reuse_distance": 1e9,
    "stage3_gather_16bit_weights_on_model_save": "auto"
  },
  "gradient_accumulation_steps": "auto",
  "gradient_clipping": "auto",
  "steps_per_print": 2000,
  "train_batch_size": "auto",
  "train_micro_batch_size_per_gpu": "auto",
  "wall_clock_breakdown": false
}
```

### Expected behavior

Gets that the embedding size is not equal to zero

Comments:
cc @stas00 

With zero-3 outside of fwd/bwd logic where this is done automatically you need to manually gather the sharded model's weights that you need. 

Please see:
https://huggingface.co/docs/transformers/main/main_classes/deepspeed#gathering-parameters

And you will find several examples in our code, e.g.:

https://github.com/huggingface/transformers/blob/88399476c3892435395618ed37993176dbb0de73/src/transformers/modeling_utils.py#L1455-L1456
Add TrOCR resources
Author: huangperry
Pull request: yes

# What does this PR do?

Adds resources of OpenAI GPT according to https://github.com/huggingface/transformers/issues/20055

Fixes #20055 (partially)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@stevhliu 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
fixed whisper positional encoding
Author: anvilarth
Pull request: yes

Whisper Positional Encoding has incorrect behavior when passing inputs_embeds:
- When we are passing `input_ids` (batch_size x seq_len) it takes the -1 dimension of it which is correct.
- When we are passing `input_embeds` (batch_size x seq_len x embedding_dim) it doesn't work. If we take -1 dimension, we get embedding dim elements.

My fix is just to take the 1 dimension which will be always correct





Comments:
_The documentation is not available anymore as the PR was closed or merged._

@clefourrier @gante

Looks like, there are no problems with tests.
![image](https://user-images.githubusercontent.com/43551010/236498753-3a084a00-873a-4c91-b9af-caf5a12481e2.png)
🌐 [i18n-KO] docs: ko: Translate `multiple_choice.mdx`
Author: gabrielwithappy
Pull request: yes

<!-- PR의 제목은 "🌐 [i18n-KO] Translated `<your_file>.mdx` to Korean" 으로 부탁드립니다 -->
# What does this PR do?

Translated the `multiple_choice.mdx` file of the documentation to Korean.
Thank you in advance for your review. 

Part of https://github.com/huggingface/transformers/issues/20179
<!-- 메인 이슈에 기록이 남아요! 가짜연구소 리포를 사용해 연습하실때는 제거해주시면 감사하겠습니다! :smile: -->

## Before reviewing
- [x] Check for missing / redundant translations (번역 누락/중복 검사)
- [x] Grammar Check (맞춤법 검사)
- [x] Review or Add new terms to glossary (용어 확인 및 추가)
- [x] Check Inline TOC (e.g. `[[lowercased-header]]`)
- [x] Check live-preview for gotchas (live-preview로 정상작동 확인)

## Who can review? (Initial)
Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

<!-- 1. 위 체크가 모두 완료된 뒤에만 가짜연구소 팀원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review? (Final)
@sgugger, @ArthurZucker, @eunseojo May you please review this PR?

<!-- 2. 가짜연구소 팀원들과 리뷰가 끝난 후에만 허깅페이스 직원들에게 리뷰 요청하는 아래 주석을 노출해주세요! -->
<!-- @sgugger, @ArthurZucker, @eunseojo May you please review this PR? -->

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Team PseudoLab, may you please review this PR? @0525hhgus, @KIHOON71, @sim-so, @gabrielwithappy, @HanNayeoniee, @wonhyeongseo, @jungnerd

@sgugger, @ArthurZucker, @eunseojo 
May you please review this PR?
Add FlaxWhisperForAudioClassification model
Author: raghavanone
Pull request: yes


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sanchit-gandhi  I have opened new PR. 


This actually broke a lot of tests on Flax Whisper, so reverting. Can you re-open the PR and rebase on main so we can see what went wrong?

@sanchit-gandhi Request you to open this PR. 


Hey @raghavanone - unfortunately a PR can't be re-opened after it's been merged. The best thing to do is add commits to the branch and create a new pull request, copying all the details over and providing a link to the original pull request. See https://stackoverflow.com/questions/12674304/github-reopening-a-merged-pull-request for details.
Whisper generation support for passing acronym to language arg
Author: connor-henderson
Pull request: no

### System Info

- `transformers` version: 4.29.0.dev0
- Platform: macOS-13.0-arm64-arm-64bit
- Python version: 3.9.16
- Huggingface_hub version: 0.12.0
- Safetensors version: 0.2.8
- PyTorch version (GPU?): 1.13.1 (False)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.5.3 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@hollance @gante

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```py
processor = WhisperProcessor.from_pretrained("openai/whisper-tiny")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-tiny")
ds = load_dataset("hf-internal-testing/librispeech_asr_dummy", "clean", split="validation")
sample = ds[0]["audio"]["array"]
input_features = processor.feature_extractor(sample, return_tensors="pt").input_features

pred_ids = model.generate(input_features, language="de")
```

Throws this error:
<img width="778" alt="Screenshot 2023-05-03 at 6 29 38 PM" src="https://user-images.githubusercontent.com/78612354/236067028-ee7ab371-e9a2-44eb-9895-b5c8f3a2fcdd.png">

Then this error when that's fixed:
<img width="1198" alt="Screenshot 2023-05-03 at 6 30 34 PM" src="https://user-images.githubusercontent.com/78612354/236067052-8f1ae574-db51-44e4-800c-aa4f38b0200e.png">


### Expected behavior

Should recognize and use language passed in acronym format as per the docstring

Comments:
cc @ArthurZucker 

Yeah I'm not sure why it was decided the language token had to be passed in there, and at the very least the current error message is misleading. Arthur is probably the best person to look at this.
Cant get deepspeed parm
Author: www516717402
Pull request: no

### System Info

latest version 2023.5.5

### Who can help?

@stas00 

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Use Accelerate and deepspeed plugin , transformers Trainer class.
I get config.ymal from `accelerate config` as follows:
``` ymal
compute_environment: LOCAL_MACHINE
deepspeed_config:
  deepspeed_config_file: /xxx/dp_zero2.json
  zero3_init_flag: false
distributed_type: DEEPSPEED
downcast_bf16: 'no'
machine_rank: 0
main_training_function: main
num_machines: 1
num_processes: 4
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false
```
`/xxx/dp_zero2.json` is config, dont care.
However get deepspeed=None from TrainingArguments  when run ` accelerate launch xxx.py `，It means deepspeed parms is invalid.


### Expected behavior

Expect `Trainer class` read deepspeed parm from ymal file

Comments:
This is a wrong repo. For Accelerate please file Issues under https://github.com/huggingface/accelerate/issues
Is there a no_trainer version for image pertaining 
Author: XZhang97666
Pull request: no
Labels: name: Good First Issue

### Feature request

I wonder if there is a run_no_trainer code for image pertaining? @NielsRogge

### Motivation

I usually use no trainer in my code because the trainer cannot should much detail.

### Your contribution

NA

Comments:
@NielsRogge

Not yet, but it would be straightforward to add.

Marking this as a good first issue.

Hi @NielsRogge, I would like to try to add it.

Hi @atturaioe, awesome.

So in [this folder](https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-pretraining), one could add a `run_mim_no_trainer.py` script, similar to the other `no_trainer.py` scripts in the examples folder.

Hi @atturaioe, are you still working on this? I would like to attempt this if you are not longer working on it.

Hi @Saad135, you can take this issues if you want, let me know.


@atturaioe Sure, I will give it a go.

Is anyone still working on this? If not I'd quite like to pick it up. 

I see from the PR that most of the work has been done but there's not been any acitivity on it recently 🤔 

@madt2709 I am working on it. Most of the work has been completed by @Saad135. His PR #20053 has been closed due to inactivity. I have taken it over in PR #23156 to complete it.
Add `no_trainer` scripts to pre-train Vision Transformers
Author: awinml
Pull request: yes

# What does this PR do?

Add scripts to pre-train Transformer-based Vision models without using the Trainer class.

Fixes #20053
Fixes #20412 
This PR completes the stalled PR #20412.

## Who can review?

@amyeroberts @NielsRogge @sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@NielsRogge As per https://github.com/huggingface/transformers/pull/20412#issuecomment-1370792519, I have made a comparison notebook running both the trainer.py and no_trainer.py scripts on a small dataset. It can be viewed [here](https://colab.research.google.com/drive/1er4n0_AoQZU-OtXZPla3i-rJ38apA_zh?usp=sharing).

Both the scripts progress similarly.

Thanks a lot @awinml! I'll assign core maintainers for a final review.
Can not find a published transformers version which fits language-modeling example.
Author: WangZhuoxian
Pull request: no

This [language-modeling example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/language-modeling)  example helps me a lot. However, the newest version so far is 4.27.4, which does not match the minimal version requirement. Looking forward to the newer version to be published. Thanks! 

Comments:
You can take the examples for v4.27 [here](https://github.com/huggingface/transformers/tree/v4.27.0/examples). The example on the main branch are compatible with the main branch of Transformers, so you need a source install.

> You can take the examples for v4.27 [here](https://github.com/huggingface/transformers/tree/v4.27.0/examples). The example on the main branch are compatible with the main branch of Transformers, so you need a source install.

Got it. Thank you!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
bridgetower model RuntimeError: The size of tensor a (865) must match the size of tensor b (325) at non-singleton dimension 1
Author: azizcu
Pull request: no

[/usr/local/lib/python3.9/dist-packages/transformers/models/bridgetower/modeling_bridgetower.py](https://localhost:8080/#) in forward(self, pixel_values)
    296         class_embeds = self.class_embedding.expand(batch_size, 1, -1)
    297         embeddings = torch.cat([class_embeds, patch_embeds], dim=1)
--> 298         embeddings = embeddings + self.position_embedding(self.position_ids)
    299         return embeddings
    300 

RuntimeError: The size of tensor a (865) must match the size of tensor b (325) at non-singleton dimension 1        when I have set max_len =128.
@abhiwand @tileintel 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Official MLflow integration
Author: BenWilson2
Pull request: no

### Feature request

Hi there maintainers, on behalf of the maintainers group for MLflow, we'd like to extend an offer to build out a full integration with MLflow. 
We're currently in the process of creating an "official" transformers flavor (a named flavor) that will support serialization and logging of components (models, feature extractors, image processors, etc) and Pipelines, as well as building support for LLM-based PIpelines for pyfunc inference (other modeling task types will be supported later). 
As part of this, we'd like to offer to expand the functionality in the current CallBack implementation from within the transformers library so that when a Trainer is instantiated, we integrate with the serialization implementation in MLflow and to handle run start context slightly differently. 
Would you all be interested in us taking a stab at this in the following weeks?

### Motivation

People love this package. Rightfully so. It's great. We'd like to make the integration with Mlflow seamless.

### Your contribution

We'd like to contribute these changes to the transformers library.

Comments:
This sounds very exciting! Happy to have a look at any PR that makes a stronger integration of MLflow!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.