noamaanMulla-03/transformers-issues huggingface dataset:
Add segmentation + object detection image processors
Author: amyeroberts
Pull request: yes

# What does this PR do?

Adds image processors for DETR, Deformable DETR, Conditional DETR, YOLOS and Maskformer, as many of the image processors methods are copied from DETR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20160). All of your documentation changes will be reflected on that endpoint.

@NielsRogge @sgugger @alaradirik Sorry for the previous issues with the docstrings. They should all be resolved now.
Add some warning for Dynamo and enable TF32 when it's set
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a warning when a user sets torchdynamo without an Ampere GPU (or higher) and also enables TF32 unless the user explicitly asked not it with `--no_tf32` to get the best performance.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Support extraction of both train and eval XLA graphs
Author: jeffhataws
Pull request: yes

Neuron supports extraction of XLA graphs for compilation. However, when both do_train and do_eval options are enabled, sizes returned by tensor operator can be 0. To avoid INVALID_ARGUMENT error, we use inequality in the check whether a tensor needs padding or not.

# What does this PR do?

This PR reduces compilation time of Hugging Face training/evaluation on Trainium using Neuron SDK.

Neuron SDK enables Hugging Face training on Trainium. To reduce compilation time, we have an optional [parallel compilation step](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/torch/torch-neuronx/api-reference-guide/training/pytorch-neuron-parallel-compile.html) which 1) extracts XLA HLO graphs by trial execution of the training script with stub graphs that output zeros only, 2) perform parallel compilations of the graphs, and 3) place compiled graphs into Neuron cache. Currently, this flow only works for do_train step in the [HF trainer API tutorial](https://awsdocs-neuron.readthedocs-hosted.com/en/latest/frameworks/torch/torch-neuronx/tutorials/training/finetune_hftrainer.html#) by itself but encounters INVALID_ARGUMENT error when do_eval is included together with do_train. 

The error during parallel compilation is due to code at https://github.com/huggingface/transformers/blob/61a51f5f23d7ce6b8acf61b5aa170e01d7658d74/src/transformers/trainer.py#L3147 that creates new tensor based on the shape of another tensor. The tensor is created but it values are zero (as opposed to the shape) during parallel compilation (trial execution of stub graphs that output zeros only).

This PR introduces an inequality in the check for whether a tensor needs padding or not. During normal execution on all platforms, the max_size is greater than or equal to the tensor size so in no cases should max_size be smaller than tensor size, except in our case where we do trial execution of stub graphs that output zeros only.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks so much @sgugger !
'WhisperTokenizer' object has no attribute 'set_prefix_tokens'
Author: nethermanpro
Pull request: no

### System Info

- `transformers` version: 4.24.0
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.27
- Python version: 3.9.15
- Huggingface_hub version: 0.11.0
- PyTorch version (GPU?): 1.13.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes

### Who can help?

@LysandreJik

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

from transformers import WhisperTokenizer
tokenizer = WhisperTokenizer.from_pretrained("openai/whisper-medium", language="spanish", cache_dir="./pretrained_models")
tokenizer.set_prefix_tokens(language="english")

AttributeError: 'WhisperTokenizer' object has no attribute 'set_prefix_tokens'

### Expected behavior

![image](https://user-images.githubusercontent.com/75082385/204983801-7b23dffc-5e52-4dca-a625-e7d9cf8e8745.png)

This method should exist according to the documentation

Comments:
Hi @nethermanpro, I'm not a maintainer, but I think I know what is going on.  

It seems that in `transformers 4.24.0`, the method  `set_prefix_tokens` is not present in this version. You can find it in this repository in the main branch, if you want to use it, you will need to install transformers directly from this repository `pip install git+https://github.com/huggingface/transformers.git` or wait for the next stable release. 

The documentation you are looking at seems to be https://huggingface.co/docs/transformers/main/en/model_doc/whisper which is the documentation of the main branch, to check the documentation of `4.24.0` you can 
select it at the top left dropdown where it says `main`. 

Here you have the link https://huggingface.co/docs/transformers/v4.24.0/en/model_doc/whisper#transformers.WhisperTokenizer



Thanks, I think that solves my problem.
Return entities extracted from the raw input for TokenClassificationPipeline 
Author: tkon3
Pull request: no

# 🚀 Feature request

Allow TokenClassificationPipeline to return entities extracted from the raw input.

## Motivation

Pretokenization (pre_tokenizer) can affect the "word" returned by the TokenClassificationPipeline. In this case `entity["word"] != sentence[entity["start"]:entity["end"]]` (see  #15785):

```python
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

MODEL_NAME = "elastic/distilbert-base-uncased-finetuned-conll03-english"
sentence = "My name is Clara and I live in Berkeley, California."

model = AutoModelForTokenClassification.from_pretrained(MODEL_NAME)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

pipe = pipeline("token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="simple")
entities = pipe(sentence)

print([(sentence[entity["start"]:entity["end"]], entity["word"]) for entity in entities])
# [('Clara', 'clara'), ('Berkeley', 'berkeley'), ('California', 'california')]
```
The pipeline should optionally return entities extracted from the input `sentence[entity["start"]:entity["end"]]`

## Who can contribute
@Narsil @LysandreJik 

Comments:
I am in favor of adding a new key `text` for this.

- That does not break BC
- `text` is already used in other places (like `ASR Pipeline`)

The reason to add this is for users that might not be easily able to capture from the original sentence:

```python
from i, tokens in enumerate(pipe(dataset)):
    ...
    # logic
    sentence = dataset[i]
    ... 
```
Now that code works only if the dataset is deterministic, in the case that is it not (like reading from a queue in various readers), then it's a bit more cumbersome to do. Making it easy is something I think is a good idea.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

bumped into this while looking for a similar issue ("f-16" is returned as "f - 16"). 
I couldn't find this option, is there any plan to include it?
Save model using save_pretrainedmethod.
Author: Iron-man-0
Pull request: no

I trained the DeTr model on custom dataset using this tutorial. 

[Niels Rogge Fine_tuning_DetrForObjectDetection](https://github.com/NielsRogge/Transformers-Tutorials/blob/master/DETR/DETR_minimal_example_(with_DetrFeatureExtractor).ipynb)

I was trying to save that model using save_pretrained method, but I'm getting a error.

Can someone please help me on how to save a model and load the same for inference using save_pretrained and from_pretrained methods. 

Thank you so much.

Comments:
Please use the [forums](https://discuss.huggingface.co/) to get help debugging your code :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[WIP] Donut flax implementation
Author: amankhandelia
Pull request: yes

This PR adds Jax support for [Donut Model](https://huggingface.co/docs/transformers/model_doc/donut).

This work is very much in progress, I need to add documentation and I am still not sure if I have added adequate number of test for the changes I have made so far, so it would be great, if someone can look up and may be provide some feedback in terms of quality of code and general direction

Things which are done
- Code is functional and passes integration test with existing Donut models in HF hub

Things which are not done
- Update documentation
- Clear list of TODOs

Fixes # (issue)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19831). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
How can I install transformers v2.11.0?
Author: FurkanGozukara
Pull request: no

### System Info

Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hello. I want to use https://github.com/xashru/punctuation-restoration

It requires specific version of transformers  v2.11.0?

First of all which specific version of Python is best for this version of transformers?

Secondly I have installed Python 3.5.4 and testing like this but it fails to install

So how can I install it?

Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32



```
C:\python355\Scripts>pip.exe install transformers==v2.11.0
DEPRECATION: Python 3.5 reached the end of its life on September 13th, 2020. Please upgrade your Python as Python 3.5 is no longer maintained. pip 21.0 will drop support for Python 3.5 in January 2021. pip 21.0 will remove support for this functionality.
ERROR: Could not find a version that satisfies the requirement transformers==v2.11.0
ERROR: No matching distribution found for transformers==v2.11.0

C:\python355\Scripts>
```

![image](https://user-images.githubusercontent.com/19240467/196563990-75a8193b-e5bc-4810-9ad4-e9b20b08e188.png)



Comments:
Hey @FurkanGozukara -- try running `pip.exe install transformers==2.11.0` (i.e. removing the `v`)

Also, please note that this is a very old version. We encourage you to use a more recent version in your project if possible!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Change doctests ci launch time
Author: ydshieh
Pull request: yes

# What does this PR do?

Current doctests CI is launched at 0h (GTM+0), but the docker images are built at  1h (GTM+0), while modeling CI is at  2h (GTM+0).

It happens a few times that we change something in doctest docker image workflow file, expect the failed tests will pass in the next run, and turns out it is not - as the next run is launched 1 hour before the new image is built.

To avoid confusion, this PR **change the doctest launch time to be the same as the modeling CI time - which is after the docker image build CI**.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix Data2VecTextForCasualLM example code documentation
Author: JuanFKurucz
Pull request: yes

# What does this PR do?

Fixes documentation of Data2VecTextForCasualLM example code, as it is currently importing `Data2VecTextTokenizer`, which does not exist, and the tokenizer is actually `RobertaTokenizer`. At the same time, the model name `"data2vec-base"` does not exist, and it doesn't particularly say to create one locally (with the change, it aims to `"facebook/data2vec-text-base"`).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
owlvit image guided detection does not work in gpu (cuda)
Author: fcakyon
Pull request: no

### System Info

- `transformers` version: 4.25.0.dev0
- Platform: Linux-5.4.0-132-generic-x86_64-with-glibc2.10
- Python version: 3.8.5
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@alaradirik @NielsRogge 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run this demo in gpu: https://huggingface.co/spaces/adirik/image-guided-owlvit

Get this error: 

```bash
  File ".../lib/python3.8/site-packages/transformers/models/owlvit/image_processing_owlvit.py", line 420, in post_process_image_guided_detection
    target_boxes = target_boxes * scale_fct[:, None, :]
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
```

### Expected behavior

I have posted a possible fix in this comment: https://github.com/huggingface/transformers/pull/20160#discussion_r1036281555

Comments:
Hi @fcakyon, thanks for bringing this up! You can expect a fix PR shortly.

As a side note, we open new issues and PRs for bugs to make it easier to track improvements. You can directly include your fix suggestions in the issue.

I will try to open a PR, give me few mins 👍 

@alaradirik tried to open the related pr here: https://github.com/huggingface/transformers/pull/20521
Fix Hubert models in TFHubertModel and TFHubertForCTC documentation code
Author: JuanFKurucz
Pull request: yes

# What does this PR do?

This PR updates the used models in the `TFHubertModel` and `TFHubertModelForCTC` example codes to the same model used in `HubertModel` and `HubertModelForCTC` other examples in the same documentation as `"facebook/hubert-base-960h"` does not exist and the actual code doesn't run.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
Author: qiuxia-alone
Pull request: no
Labels: name: bug

### System Info

macos
pycharm
python                            3.7
transformers                  4.20.1
torch                               1.12.0
torch-scatter.                 2.0.9
tensorflow                      2.3.0


### Who can help?

@NielsRogge

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

when i run `from transformers import TapasForQuestionAnswering`, then i meet error: Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)


### Expected behavior

expect nothing happened

Comments:
It's likely that you have a mismatch between your torch and torch-scatter installs. Can you import torch-scatter on its own without the SIGSEGV?


> It's likely that you have a mismatch between your torch and torch-scatter installs. Can you import torch-scatter on its own without the SIGSEGV?

hi, i have installed the right version of torch-scatter(2.0.9) as showed in [https://pytorch-geometric.com/whl/](url), but i import torch_scatter as you said and met the SIGSEGV.

sorry, i get the wrong version of torch, the problem solved, thank you for your apply.

Great, happy you could solve the problem!

Having faced the same issues and also fixed it by updating the version of pytorch
add timesformer model
Author: fcakyon
Pull request: no
Labels: name: New model

### Feature request

Add [TimeSformer, ICML 2021](https://arxiv.org/abs/2102.05095) model to HuggingFace transformers library.

Source code: https://github.com/facebookresearch/TimeSformer
Weight files: https://github.com/facebookresearch/TimeSformer#model-zoo
Alternative implementation: https://github.com/lucidrains/TimeSformer-pytorch

### Motivation

[TimeSformer, ICML 2021](https://arxiv.org/abs/2102.05095) is a transformer-based video understanding model from META AI, and all recent video classification/action recognition papers compare their proposal with this model.

As a Ph.D. candidate researching video transformers, I want to add TimeSformer support to my [video-transformers](https://github.com/fcakyon/video-transformers) package, but there is no easy way of doing that. It would be super easy if HuggingFace included it.

It would be an excellent addition to the `transformers` library considering it has 400+ citations and is an essential benchmark model.

### Your contribution

As a Ph.D. candidate working on video transformers, I would like to work on adding this architecture to the HuggingFace.

I want to start a PR but don't know where to start. Should I take the VideoMAE model as a reference since it has similar input/output to TimeSformer?

### Open source status
- [x] The model implementation is available
- [x] The model weights are available

Comments:
cc @NielsRogge!

Hi,

Sure, let me know your email address, then we can set up a slack channel for easier communication. 

> Hi,
> 
> Sure, let me know your email address, then we can set up a slack channel for easier communication.

I have written my mail to you by Discord message 👍
fix cuda OOM by using single Prior
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Fixes the OOM issue with the `5b`  model and `fp16` sampling. 
Also fixes the slow generation test by sending each prior to `cuda` only when they are actually used. 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

This will require #20485 to be merged, otherwise the logits for `fp16_sampling` will differ. 

I will  wait the mentioned PR #20485 :-) then back to this one. Also see my comment in that PR 🙏 @ArthurZucker 🙏 

I'll just push to model to another repo and ping you back here!
remove truncation in whisper
Author: ydshieh
Pull request: yes

# What does this PR do?

remove truncation in whisper

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @ArthurZucker ,

Just got an error, seems related to this issue?

`RuntimeError: The size of tensor a (507) must match the size of tensor b (448) at non-singleton dimension 1`

@RK-BAKU  This PR is not merged yet. Are you trying this PR instead of a stable release or the `main` branch?

Hey! @RK-BAKU  Could you provide a reproducing script and open a separate issue ?