noamaanMulla-03/transformers-issues huggingface dataset:
[`BiT`] Small patch fix
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes a tiny issue that you can encounter if you load `BiT` in fp16.
`diffusers` uses this model under the hood for Depth Estimation inpainting and users get this error:
```
    593 
    594         layer_dropouts = [
--> 595             x.tolist() for x in torch.linspace(0, config.drop_path_rate, sum(config.depths), dtype=torch.float32).split(config.depths)
    596         ]
    597 

RuntimeError: "linspace_cpu" not implemented for 'Half'

```
However on `diffusers` side this can be fixed by installing `accelerate` and load the pipeline with `low_cpu_mem_usage=True`. But better to fix it to avoid any misleading issue


cc @sgugger @patil-suraj 

Otherwise to reproduce:
```
import torch
from transformers import BitModel

model = BitModel.from_pretrained("google/bit-50", torch_dtype=torch.float16)
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add XCiT Model
Author: IMvision12
Pull request: no
Labels: name: New model

### Model description

Cross-covariance image transformer (XCiT) is built upon XCA. It combines the accuracy of conventional transformers with the scalability of convolutional architectures. [Paper](https://arxiv.org/abs/2106.09681)

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Official Implementation: https://github.com/facebookresearch/xcit
Timm Implementation: https://github.com/rwightman/pytorch-image-models/blob/main/timm/models/xcit.py

Comments:
Fix load from PT-formatted checkpoint in composite TF models
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the slow test `TFViT2GPT2EncoderDecoderModelTest::test_real_model_save_load_from_pretrained` which was broken by the new `safetensors` integration. The main problem was that this model loads a GPT-2 as its decoder, which has a safetensors checkpoint formatted in a PyTorch-like format, and that model was loaded with wrong weight names.

Moving the variable scope code before we try to load PyTorch-like checkpoints fixes the issued.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Run shellcheck on all *.sh scripts and attempt to fix errors
Author: tripleee
Pull request: yes

Also, refactor a few repetitive code patterns

# What does this PR do?

Attempt to fix shell scripting errors in examples etc.

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [n/a] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [n/a] Did you write any new necessary tests?


## Who can review?

- maintained examples (not research project or legacy): @sgugger, @patil-suraj


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19979). All of your documentation changes will be reflected on that endpoint.

Thanks. I'm hoping it could be accepted simply to give users of the code base better examples to copy/paste from; the changes are mainly mechanical.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Enables torchrun for XLA-based accelerators
Author: jeffhataws
Pull request: yes

# What does this PR do?

This PR enables torchrun for XLA-based accelerators (TPU/NeuronCore) by using torch.distributed XLA backend. It is dependent on the torch/xla change https://github.com/pytorch/xla/pull/3609.

Example application is the AWS Neuron tutorial with HF Trainer that uses torchrun:

https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/torch/torch-neuronx/tutorials/training/finetune_hftrainer.html

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19907). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Migrate torchdynamo to torch.compile
Author: sgugger
Pull request: yes

# What does this PR do?

This PR migrates the current integration with PyTorch 2.0 to use the entry point they introduced: `torch.compile`. As a consequence, the `torchdynamo` argument is deprecated to the profit of `torch_compile_backend` and `torch_compile_mode`. Setting either will trigger a model compilation.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Whilelist Transformers private method in DummyObject
Author: sgugger
Pull request: yes

# What does this PR do?

Fixes #20671

As reported in #20671, calling `AutoModel.from_config` on a model with a missing specific soft dependency does not raise the appropriate error. This is because this method ends up calling `ModelClass._from_config` and the `DummyObject` class does not raise the error on all private attributes (basically because we need the `__xxx__` attribute to stay the same). This PR whitelists `_from_config` to fix the issue.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

You can't do that kind of check without triggerring recursion errors (cause it calls methods like `__getattribute__` inside that method ;-) ).
Calling `AutoModel.from_config()` method for a model requiring timm does not raise ImportError although it should
Author: fxmarty
Pull request: no

### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
- Python version: 3.9.12
- Huggingface_hub version: 0.11.0.dev0
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): 2.9.1 (True)
- Flax version (CPU?/GPU?/TPU?): 0.5.2 (cpu)
- Jax version: 0.3.14
- JaxLib version: 0.3.14
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`pip uninstall timm`, and then:

```python
from transformers import AutoModel, AutoConfig

cfg = AutoConfig.from_pretrained("hf-internal-testing/tiny-random-detr")
model = AutoModel.from_config(cfg)
```

raising:
```
Traceback (most recent call last):
  File "<tmp 1>", line 18, in <module>
    model = AutoModel.from_config(cfg)
  File "/home/fxmarty/hf_internship/transformers/src/transformers/models/auto/auto_factory.py", line 410, in from_config
    return model_class._from_config(config, **kwargs)
  File "/home/fxmarty/hf_internship/transformers/src/transformers/utils/import_utils.py", line 1008, in __getattribute__
    return super().__getattribute__(key)
AttributeError: type object 'DetrModel' has no attribute '_from_config'
```

### Expected behavior

It should raise:

```
ImportError: 
DetrModel requires the timm library but it was not found in your environment. You can install it with pip:
`pip install timm`. Please note that you may need to restart your runtime after installation.
```

as in https://github.com/huggingface/transformers/blob/main/src/transformers/utils/dummy_timm_and_vision_objects.py#L78

Comments:
Indeed, I can see why and it's an easy fix. Will make a PR in a couple of hours!

Can you try the PR mentioned above?

Works well thanks for the fix!
[`ViTHybrid`] Fix `accelerate` slow tests
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes failing `ViTHybrid` `accelerate` tests

Uses the same procedure as DPTHybrid for `backbone_featmap_shape` - Now all slow tests should pass :-) 

cc @sgugger @ydshieh 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
openai whisper ASR pytorch to tflite
Author: nyadla-sys
Pull request: no
Labels: name: New model

### Model description

I'm trying to figure out how to create tflite models(int8/float32 ) for OpenAI->Whisper ASR model (Tiny.en.pt)
Somehow below generated tflite file getting crashed while running inference 
https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/tinynn_pytorch_to_tflite_int8.ipynb



### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
@patrickvonplaten @sgugger @amyeroberts @ArthurZucker :Could anyone of you help me on this ?

Below notebopok generates tflite file ,however i have not validated with real speech input 
https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/tflite_from_huggingface_whisper.ipynb

Can anyone help me to run inference on this notebook to validate generated tflite file 

@ydshieh Could you please help me to validate  tflite file that got generated using below google notebook 
https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/tflite_from_huggingface_whisper.ipynb

@nyadla-sys The TF Whisper is just released, maybe you can try with it.

However, our team is still working on some TensorFlow model saving issues, so I don't know if the conversion/inference will work out of the box.

@ydshieh ran inference on converted tflite file and it doesn't work.

@ydshieh 
I could generate encoder int8 tflite model from openai->whisper(pytorch) and ran inference.
https://colab.research.google.com/github/usefulsensors/openai-whisper/blob/main/notebooks/generate_tflite_from_whisper.ipynb
can someone help me to generate int8 decoder tflite model from openai->whisper(pytorch) and run inference?
Add deprecation warning when image FE instantiated
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds a deprecation warning if someone tries to create a vision feature extractor. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
beam_search 中记录 beam_indices 的问题
Author: jicoder-nwpu
Pull request: no

### System Info

some beam_indices become 0

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

if return_dict_in_generate and output_scores:
     beam_indices = tuple((beam_indices[beam_idx[i]] + (beam_idx[i],) for i in range(len(beam_indices))))

### Expected behavior

if return_dict_in_generate and output_scores:
     beam_indices = tuple((beam_indices[i] + (beam_idx[i],) for i in range(len(beam_indices))))

Comments:
Please follow the template of the issue. No one can help you without a description of the problem and a clear reproducer.
NER example
Author: jiaqianjing
Pull request: no

https://github.com/huggingface/transformers/blob/9e56aff58a742b48fc8edea8d28d5b80330efbcc/examples/pytorch/token-classification/run_ner_no_trainer.py#L601
I think modifying it this way would be better.
```python
    true_predictions = [
        [label_list[p] for (p, l) in zip(prediction, label) if p != -100]
        for prediction, label in zip(predictions, labels)
    ]
```

Comments:
Added TFBartForSequenceClassification
Author: IMvision12
Pull request: yes

# What does this PR do?

Fixes: #19653


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger I have tested locally, not sure why tests are failing

@IMvision12 the common tests do not pass for the model you have added, you can run them with `pytest tests/models/bart/test_modeling_tf_bart.py`.

Also the equivalence test PyTorch/TensorFlow does not pas either.

@sgugger I've looked at some SequenceClassification models and the tests that have been written but none of them have TFSeq2SeqSequenceClassification or tests for it, so I'm not sure how to add a test for that. Need little help

@ydshieh Still the tests are failing

This issue has been solved in this PR here #20570 so i am closing this PR
Long-form (including timestamps) for whisper
Author: JeffreyWardman
Pull request: no

### Feature request

https://github.com/huggingface/transformers/commit/504cd71a6b172f177e6da513bea94fadb18ad99c
- Inference is currently only implemented for short-form i.e. audio is pre-segmented into <=30s segments. Long-form (including timestamps) will be implemented in a future release.

When would the ETA be for this?

### Motivation

Whisper is not usable for long audio of speech, or for chunking audio based on timestamps determined by the ASR.

### Your contribution

Guidance/PR in longer term future if not picked up by others in the next month or so

Comments:
cc @sanchit-gandhi and @ArthurZucker 

Hey @JeffreyWardman! I believe @ArthurZucker has started looking into this, see https://github.com/huggingface/transformers/issues/19490#issuecomment-1285166541 for context!

Thanks @sanchit-gandhi! By the looks of it, it would still be missing the timestamps. This is quite an important feature for me. I'm not completely familiar with the underlying code for huggingface. How does the chunking work? Does it calculate the first break between words after a given duration?

cc @ArthurZucker who knows more about timestamp generation!

This blog highlights quite nicely how chunking works in Transformers: https://huggingface.co/blog/asr-chunking

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Does whisper implementation of hugging support timestamps to generate SRT files like openai/whisper implementation?

https://github.com/openai/whisper/blob/main/whisper/utils.py#L64

Not yet! Working on this you can follow #20620 !
Add Whisper large V2 model
Author: mrm8488
Pull request: no
Labels: name: New model

### Model description

It seems openAI has just released a V2 of its large Whisper model.
- The "large-v2" model is trained for more epochs with regularization and shows improved performance compared to the previous large.
- It has the same architecture as the original large model.
- When `load_model("large")` is called, the "large-v2" model will be loaded.
More, here: https://github.com/openai/whisper/commit/4179ed2475cc84cba66868b516232ef1b74dacdf

I can upload it to the hub. Cc: @younesbelkada @patrickv

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
cc @ArthurZucker 

Hey! The `large-v2` is already converted, we are just waiting for OpenAI's approval for the release 😉 

Cool!

FYI : https://huggingface.co/openai/whisper-large-v2 

Not really sure if we want to change the `large` for `large-v2`, not really backward compatible
Change transformers.onnx to use optimum.exporters.onnx
Author: michaelbenayoun
Pull request: yes

# What does this PR do?

As the title say. The `transformers.onnx` command-line tool now uses the `optimum.exporters.onnx` command-line tool in the background, and redirects the user to use this tool directly for the next times (same in the documentation).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
skip `test_multi_gpu_data_parallel_forward` for `MaskFormerSwinModelTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

`MaskFormerSwinModel` outputs `hidden_states_spatial_dimensions`, which is ``tuple(tuple(int, int))``, and can't be collected by `nn.DataParallel` to form a final output value.

(When I remove this attribute, this test passes.)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Vision processors - replace FE with IPs
Author: amyeroberts
Pull request: yes

# What does this PR do?

Replaces feature extractors with image processors in the `Processor` class which bundle together tokenizers and feature extractor. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix bug: replace left over FE references
Author: amyeroberts
Pull request: yes

# What does this PR do?

Fixes failing Flava processor tests introduced by #20590 - replaces any exisiting feature extractor reference with image processors. 


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Training ConformerCTC suitable for online inference
Author: pfeatherstone
Pull request: no

### Feature request

Provide a mechanism (or docs if already possible) for training a Conformer model with CTC loss function, such that when inferring live using blocked buffered data, you get the same output as if passing the whole data in one go. Also, discuss if this is resilient to sample offsets.

### Motivation

I would like to use a Conformer model trained with CTC loss live using buffered data coming of a sensor.

### Your contribution

Maybe if i had guidance

Comments:
Please use the [forums](https://discuss.huggingface.co) for such questions.

Ok sorry

@sgugger I added a post [here](https://discuss.huggingface.co/t/conformerctc-for-streaming/27480). i would greatly appreciate your input.