noamaanMulla-03/transformers-issues huggingface dataset:
feat: pipeline registry for supporting custom pipelines
Author: aarnphm
Pull request: no

### Feature request

I propose a simple registry abstraction to allow users to dynamically register custom pipelines to transformers.

```python
audio_classification_tmpl = {
        "impl": AudioClassificationPipeline,
        "tf": (),
        "pt": (AutoModelForAudioClassification,) if is_torch_available() else (),
        "default": {"model": {"pt": "superb/wav2vec2-base-superb-ks"}},
        "type": "audio",
    }

PipelineRegistry.register_pipeline("audio-classification", audio_classification_tmpl)
```

A pseudo example for `PipelineRegistry` implementation:
```python
class PipelineRegistry:
    SUPPORTED_TASKS: dict[str, dict[str, PiplineBase | dict[str, Any]]

    @classmethod
    def registry_pipeline(cls, task: str, task_metadata: dict[str, Any]:
        cls.SUPPORTED_TASKS[task] = task_metadata
```

For any custom pipeline user can simply do
```python

from transformers.pipelines import PipelineRegistry

my_custom_task_tmpl = {
        "impl": CustomPipeline,
        "tf": (),
        "pt": (AutoModelForAudioClassification,) if is_torch_available() else (),
        "default": {"model": {"pt": "my_custom_wav2vec"}},
        "type": "custom",
    }

PipelineRegistry.register_pipeline("custom-task", my_custom_task_tmpl)
```

### Motivation

Currently, pipelines abstraction provides users with quick and easy way to run any given tasks. However, it is very difficult to create and adding support for custom pipelines. According to [docs](https://huggingface.co/docs/transformers/add_new_pipeline#adding-it-to-the-list-of-supported-tasks), If users want to add a new pipeline, one would have to come in and modify `transformers` source code. 

This is often less than ideal. It would be nice for pipeline to have a "registry" abstraction where transformers can allow users to register their custom pipeline to transformers without the hassle of editing the source code.

### Your contribution

#17905 

Comments:
Mrbean/codegen onnx
Author: sam-h-bean
Pull request: yes

# What does this PR do?
Codegen was added with an ONNX config but not with the model added to the features manager so trying to actually export an ONNX config is failing.

```bash
11497 ± RUN_SLOW=1 pytest tests/onnx/test_onnx_v2.py -k "codegen" -v
===================================================================================== test session starts ======================================================================================
platform darwin -- Python 3.9.10, pytest-7.1.2, pluggy-1.0.0 -- /Users/marklar/workspace/transformers/venv/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/Users/marklar/workspace/transformers/.hypothesis/examples')
rootdir: /Users/marklar/workspace/transformers, configfile: setup.cfg
plugins: xdist-2.5.0, forked-1.4.0, timeout-2.1.0, hypothesis-6.47.0, dash-2.5.0
collected 371 items / 367 deselected / 4 selected                                                                                                                                              

tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_029_codegen_causal_lm PASSED                                                                                       [ 25%]
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_030_codegen_default PASSED                                                                                         [ 50%]
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_on_cuda_029_codegen_causal_lm PASSED                                                                               [ 75%]
tests/onnx/test_onnx_v2.py::OnnxExportTestCaseV2::test_pytorch_export_on_cuda_030_codegen_default PASSED                                                                                 [100%]

```


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patil-suraj @JingyaHuang 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
rust_model.ot not saved by save_pretrained()
Author: Zacchaeus00
Pull request: no
Labels: name: bug

### System Info

I am running my experiments on a machine without internet, so I need to save the tokenizers first. However, I found that with the `save_pretrained()` method, the `rust_model.ot` is not saved to the directory. The model is `deberta-v3-base` and the version is 4.20.1.

### Who can help?

@LysandreJik @SaulLu

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
ckpt = 'microsoft/deberta-v3-base'
tokz = AutoTokenizer.from_pretrained(ckpt, use_fast=True)
model = AutoModel.from_pretrained(ckpt)
tokz.save_pretrained(ckpt)
model.save_pretrained(ckpt)
$ls
> added_tokens.json  pytorch_model.bin	    spm.model
> config.json	   special_tokens_map.json  tokenizer_config.json
```

### Expected behavior

rust_model.ot saved by save_pretrained()

Comments:
Rust models are not supported by `transformers`. I believe the models you're talking about are the ones coming from @guillaume-be's [rust-bert](https://github.com/guillaume-be/rust-bert) repository.

@LysandreJik Thanks. I got it. I was thinking about https://huggingface.co/microsoft/deberta-v3-base/blob/main/rust_model.ot.
Grammatically updated the Readme file
Author: shivagowri1928
Pull request: yes

# What does this PR do?
grammatically updated the readme file
<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people. @sgugger 

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

okay thankyou
-1e9 constants in T5 implementation
Author: marhlder
Pull request: no

These -1e9 constants are too large for fp16 training

https://github.com/huggingface/transformers/blob/df735d1317994e366ab0edff6c55930e18912b7c/src/transformers/models/t5/modeling_tf_t5.py#L728
https://github.com/huggingface/transformers/blob/df735d1317994e366ab0edff6c55930e18912b7c/src/transformers/models/t5/modeling_tf_t5.py#L746

Maybe they should be made configurable?

Comments:
Probably of interest to @ydshieh
Update expected values in CodeGen tests
Author: ydshieh
Pull request: yes

# What does this PR do?

Update expected values in CodeGen test `test_codegen_sample`. The currently value works for other GPU, but for Nvidia T4, we need the values in this PR. 

Note that `do_sample` will call `self.sample` (in `generatioin_utils.py`) which uses `torch.multinomial`, which is not 100% reproducible across different accelerators.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

kindly ping @patil-suraj as I am eager toward 0 test failure 🚀 on CI report
fixing fsdp autowrap functionality
Author: pacman100
Pull request: yes

### What does this PR do?
1. PyTorch has renamed default_auto_wrap_policy to size_based_auto_wrap_policy. This PR updates the same.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks for the fix. I think we should proceed differently to still support the previous nightly builds, and import the old name as the new name then.

Hello, I have updated the version requirement to the stable torch version `1.12.0` and this version has the updated name for the function. The support in `1.11.0` couldn't allow for saving model when using FSDP.
Fix FlaxBigBirdEmbeddings
Author: ydshieh
Pull request: yes

# What does this PR do?

Current `FlaxBigBirdEmbeddings` applies layer norm before dropout, while `BigBirdEmbeddings` and Google's original `BigBird`
applies dropout first. This PR fixes this inconsistency.

Flax
(layernorm --> dropout)

https://github.com/huggingface/transformers/blob/6f29029b05df221c0c37fd2e87aeadc9cb6ce5d7/src/transformers/models/big_bird/modeling_flax_big_bird.py#L232-L233

PyTorch
(dropout immediately after embedding)
https://github.com/huggingface/transformers/blob/6f29029b05df221c0c37fd2e87aeadc9cb6ce5d7/src/transformers/models/big_bird/modeling_big_bird.py#L311-L312

Google
(dropout immediately after embedding)
https://github.com/google-research/bigbird/blob/5f2a5aa7fbab23e32e0e0b41c5f0192f0c023e05/bigbird/core/utils.py#L565-L566

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I will merge this PR today.
check min version
Author: milad1378yz
Pull request: no
Labels: name: bug

### System Info

```shell
Hi, in transformers/examples/pytorch/text-classification/run_glue.py in line 50, there is a line of code :
'''
check_min_version("4.20.0.dev0")
'''
and there is no 4.20.0 version so I think it should be corrected.
```


### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

python transformers/examples/pytorch/text-classification/run_glue.py \
  --model_name_or_path bert-base-cased \
  --train_file ./train.csv  \
  --validation_file ./test.csv \
  --do_train \
  --do_eval \
  --evaluation_strategy epoch \
  --max_seq_length 128 \
  --per_device_train_batch_size 32 \
  --learning_rate 5e-5 \
  --max_seq_length 128 \
  --num_train_epochs 2 \
  --seed 2021\
  --output_dir /yazdani/tmp/imdb/

### Expected behavior

```shell
change this part to 
'''
check_min_version("4.18.0.dev0")
'''
```


Comments:
Hello! The `4.20.dev0` version means that it's the current `main` branch. The recommended way to run these examples is to clone the repository.

See the following note: https://github.com/huggingface/transformers/tree/main/examples#important-note

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Allow creation of tokenizer from a vocab dictionary
Author: itaihay
Pull request: no

### Feature request

Tokenizers that need a vocab.json file are expected to be created in this manner:
`transformers.Wav2Vec2CTCTokenizer(vocab_file="path/to/vocab.json")`
With the file read with a `json.load` statement.

I'm suggesting an optional  `vocab` parameter  that could be passed instead.

```
vocab= {"a":0, "b":1,......}
tokenizer = transformers.Wav2Vec2CTCTokenizer(vocab=vocab)

# Would also be possible
transformers.Wav2Vec2CTCTokenizer(vocab_file="path/to/vocab.json")
````

### Motivation

Remove the necesity to clutter the disk with a vocab file and allow a dynamic vocab creation process

### Your contribution

I could implement this if it seems like a good addition

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Adding a ISSUE_TEMPLATE for the translation of docs
Author: omarespejel
Pull request: no

### Feature request

Users can create issues to translate the docs to several languages. This technique worked for the translation of [the course](https://github.com/huggingface/course/issues) (cc @lewtun).

Since we have [several docs to translate](https://github.com/huggingface/transformers/issues/15947) a template would be adequate. 

Currently, the [ISSUE TEMPLATES](https://github.com/huggingface/transformers/tree/main/.github/ISSUE_TEMPLATE) of the Transformers library are .yml. However, I would prefer to write a PR with a MD (similar to the [one in the Course](https://github.com/huggingface/course/blob/main/.github/ISSUE_TEMPLATE/translations.md)). We do not need to ask info from the issue writer, maybe a field to ask if they are willing to take leadership of the translation they are proposing.

### Motivation

Allowing the users to create their own issues (and possibly take ownership/leadership) would allow for a faster translation.

### Your contribution

If this is accepted I can create a PR with the issue template.

Comments:
I closed the issue for the moment.

This issue was mentioned in a previous community issue #17404 for translating into Italian 🇮🇹.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Ensure PT model is in evaluation mode and lightweight forward pass done
Author: amyeroberts
Pull request: yes

Small update to the `pt-to-tf` CLI. Sets the pytorch model into evaluate model and uses `no_grad` context to make the memory requirements lighter. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Restore original task in test_warning_logs
Author: ydshieh
Pull request: yes

# What does this PR do?

`test_warning_logs` modifies `PIPELINE_REGISTRY` via `PIPELINE_REGISTRY.register_pipeline(alias, {})`, which fails the tests in `TextClassificationPipelineTests`.

This PR restores the original task at the end of `test_warning_logs`.

cc @aarnphm

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Shifting labels for causal LM when using label smoother
Author: seungeunrho
Pull request: yes

# What does this PR do?

Fixes #17960 

When training CausalLM such as GPT2, loss is computed within model's `foward()` function and `labels` are shifted internally. However, if label smoothing is applied, loss is computed in trainer's `compute_loss` function and `labels` are not shifted. This causes misalignment of `labels` and corresponding `input_ids`. This commit is for resolving this misalignment.




## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Training loss doesn't decrease on TPU while works fine on GPU
Author: AbuUbaida
Pull request: no

For the summarization task, where I train with an encoder-decoder model on GPU, it works fine and the loss gets lower over iterations. But when I change the device to `device = xm.xla_device()` and optimizer to `xm.optimizer_step(optimizer, barrier=True)` on single-core TPU, the training loss remains nearly unchanged!!

**Here's the reproducible code:** https://colab.research.google.com/drive/1pC2CF3ipWt0eJrfXdznwAZD3zs0sX1kd?usp=sharing
Is it a bug or I am missing something?

Comments:
Maybe @sgugger has an idea or knows someone who does!

This is more of a question for the PyTorch XLA folks, since you're not using any of our tools for training.

> This is more of a question for the PyTorch XLA folks, since you're not using any of our tools for training.

Thanks I have asked them and here's the solution: https://github.com/pytorch/xla/issues/3675#issuecomment-1171702988

Ah yes, I missed it but it's indeed a common mistake on TPUs!
(FYI: by using Accelerate to power your training loop, your mistake would have been automatically fixed ;-) )

Nice to know! Thanks for the information @sgugger. Actually, I started with the HF Trainer but facing [this](https://github.com/huggingface/transformers/issues/14989#issue-1091070983) issue I moved to [this](https://github.com/huggingface/transformers/issues/14989#issuecomment-1003349939) solution which used PyTorch loop instead though I am using v4.18.0.
Fix wrong image conditional checking
Author: Charlyo
Pull request: yes
Labels: name: Feature extractors

Apparently, If an empty batch comes in, it is considered valid. However, then line 138 brakes since it tries to access element 0 of an empty batch.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16927). All of your documentation changes will be reflected on that endpoint.

Hi,

Thanks for your PR. If we go ahead with this, then it should also be fixed for all other feature extractors, like ViT, BEiT, DeiT, etc. cc @patil-suraj 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Gently pinging @Charlyo, let me know if you want to finish this, otherwise happy to take over :) 

Sorry guys! Feel free to take over @patil-suraj  :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Large model loading: add link to existing documentation
Author: LysandreJik
Pull request: yes

The documentation for large model loading is in two different places. This adds a link from one to the other, showing the auto device map.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hmmm yes, I agree, but this is still linking to `from_pretrained`, just a part of the documentation that contains more information about the `low_cpu_mem_usage`. The part I have removed glosses over it quickly, whereas the part I link to has extensive documentation covering both `low_cpu_mem_usage` and the `device_map` argument to pass to `from_pretrained`.

Reading the documentation right now, if we're interested in big models and we open the "Instantiating a big model" page in the toctree, there are no mention of the `device_map`. This is what this PR aims to fix.