noamaanMulla-03/transformers-issues huggingface dataset:
Replace reduce_labels with do_reduce_labels
Author: amyeroberts
Pull request: yes

# What does this PR do?

The `reduce_labels` flags for most of the image processors was deprecated for`do_reduce_labels`. This was to keep consistent with the `do_xxx` pattern with other flags.  

This PR deprecates the flag for any models that still used `reduce_labels`. 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update tests: replace feature extractor tests with image processor
Author: amyeroberts
Pull request: yes

# What does this PR do?

Replaces all feature extractor references with image processor references in the `test_image_processing_xxx.py` files
    * `feature_extractor = XxxFeatureExtractor()` -> `image_processor = XxxImageProcessor`

Requires https://github.com/huggingface/transformers/pull/20785 in order to replace `FeatureExtractionSavingTestMixin`

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Models docstring
Author: sgugger
Pull request: yes

# What does this PR do?

This PR cleans up all docstrings following up from #20757 and #21199. It removes the need for the `processor_class` in TensorFlow and Flax generic examples by setting in the examples like #20757 did for PyTorch then makes a full pass across all models to clean up the docstrings (removing the processor_class` in the `add_code_sample` decorator, remove random outputs, use the auto classes for preprocessing).

Note that in some cases we can't use the auto-classes for preprocessing: when linking to the `__call__` method of a processor or image processor, we need the actual class (cc @amyeroberts I changed a couple of things you did here).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you @sgugger for cleaning this up. With all ~250 files, I will trust you instead of look lines by lines, except one question below.

I would definitely prefer to run a doctest first offline before merging this PR - for which I can launch on my side. From previous PRs, it has shown there are always some surprise. I will launch doctest CI when all reviewers give their approval.

**So here my question**

> Note that in some cases we can't use the auto-classes for preprocessing: when linking to the __call__ method of a processor or image processor, we need the actual class (cc @amyeroberts I changed a couple of things you did here).

I see even in such places, we still have
```python
Pixel values can be obtained using [`AutoImageProcessor`].  See [`ConvNextImageProcessor.__call__`] for details.
```
I don't have much context and prior knowledge, but is it true we want to use `AutoImageProcessor` but `ConvNextImageProcessor.__call__` in such cases?

> With all ~250 files, I will trust you instead of look lines by lines.

A review would still be much appreciated, as it could catch accidental typos.

> I would definitely prefer to run a doctest first offline before merging this PR - for which I can launch on my side. From previous PRs, it has shown there are always some surprise. I will launch doctest CI when all reviewers give their approval.

Sure, we can wait for that as long as the results are available before the release branch is cut.

> I don't have much context and prior knowledge, but is it true we want to use AutoImageProcessor but ConvNextImageProcessor.__call__ in such cases?

Yes.

I triggered the doctest CI against the (last) commit (so far) in this PR. Will take a look on the PR changes too :-)

[run page](https://github.com/huggingface/transformers/actions/runs/3987623228/jobs/6837694181)
Skip `test_multi_gpu_data_parallel_forward` for `UperNetModelTest`
Author: ydshieh
Pull request: yes

# What does this PR do?

Same as `BEIT`, this model uses some `add_module` and we need to skip this test.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Wait, are you saying that models that using add_module can't leverage multi-GPU training?

> Wait, are you saying that models that using add_module can't leverage multi-GPU training?


(well, just not with the way we used in `test_multi_gpu_data_parallel_forward`, which uses `nn.DataParallel`)
(PyTorch says `It is recommended to use [DistributedDataParallel] instead)

**Partially** , that's my observation while debugging `Maskformer`, `BEIT`, and then with `LayoutLMV2`, `Data2VecVision` etc.

```
    @unittest.skip(
        reason="Data2VecVision has some layers using `add_module` which doesn't work well with `nn.DataParallel`"
    )
```

see https://github.com/huggingface/transformers/pull/17864



@NielsRogge Let me know if you have further question :-) before giving 👍✅ . Thank you 🙏
Text generation link not working
Author: BramVanroy
Pull request: no

I'm not sure if this is the right place to notify such issues.

On [this page](https://huggingface.co/docs/transformers/main_classes/text_generation) the link in the second paragraph to "text generation strategies guide" does not work. It should point to: https://huggingface.co/docs/transformers/main/generation_strategies

Comments:
will move this to `transformers`

Thanks for noticing! should be addressed in #21276
[Doc] fix broken link
Author: younesbelkada
Pull request: yes

Fixes https://github.com/huggingface/transformers/issues/21275 
I can confirm the link at least works in https://moon-ci-docs.huggingface.co/docs/transformers/pr_21276/en/main_classes/text_generation

cc @ydshieh 💯 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Use return_tensors="np" instead of "tf"
Author: Rocketknight1
Pull request: yes

This PR is doing exactly the same thing as the [notebooks PR here](https://github.com/huggingface/notebooks/pull/308).

In our TF examples, we use return_tensors="tf" for the data collators. However, `prepare_tf_dataset` and `to_tf_dataset` actually use a NumPy loader internally, which we wrap with a `tf.data.Dataset` at the end. As a result, return_tensors="np" works much better for them, and avoids some weird slowdown bugs we've experienced.

This PR replaces every instance in our examples with return_tensors="np". (cc @gante, @amyeroberts, @sayakpaul just so you're aware)


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Does "kwargs" actually work in from_pretrained of Tokenizer?
Author: vapavlo
Pull request: no

### System Info

Colab
transformers==4.25.1


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Description of the parameters for the method **from_pretrained** states: 
```
kwargs (additional keyword arguments, *optional*):
                Will be passed to the Tokenizer `__init__()` method. Can be used to set special tokens like
                `bos_token`, `eos_token`, `unk_token`, `sep_token`, `pad_token`, `cls_token`, `mask_token`,
                `additional_special_tokens`. See parameters in the `__init__()` for more details.
```
However, the following code does not change parameters of the tokenizer:
```
tokenizer = transformers.AutoTokenizer.from_pretrained(
    'bert-base-uncased', 
    kwargs={'model_max_length': 777, 'cls_token': '[CLASS]'}
)
print(tokenizer.model_max_length)
print(tokenizer.cls_token)
```
Output:
```
512
[CLS]
```
Why is that?


### Expected behavior

The output I want to see is:
```
777
[CLASS]
```


Comments:
How to finetune MBART on an single language?
Author: BakingBrains
Pull request: no

Hello, 
Can anyone please suggest, how can I finetune MBART for a specific language.

I found this Asian Bert repo. https://github.com/hyunwoongko/asian-bart
where they used MBART (using [mBart](https://arxiv.org/abs/2001.08210) by embedding layer pruning) for single language.
I want to do the same.

I am unable find any good resource on this. Any suggestions?

Thanks and Regards.

Comments:
Please use the [forums](https://discuss.huggingface.co/) for question like this, the whole community will be there to help! We keep issues for bugs and feature requests only :-)

@sgugger Thank you
I asked the same in forum, but no response there. If anyone know here, can suggest.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ImportError while trying to get the OS and software versions
Author: vaibhav-k
Pull request: no

### System Info

OS: Ubuntu 22.04; Python version: Python 3.10.6

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Simply run the command `python3 src/transformers/commands/transformers_cli.py env`.

### Expected behavior

I wanted to get the OS and software versions but instead got the error
```
Traceback (most recent call last):
  File "/home/skywalker/Downloads/transformers/transformers/src/transformers/commands/transformers_cli.py", line 18, in <module>
    from .add_new_model import AddNewModelCommand
ImportError: attempted relative import with no known parent package
```

Comments:
The CLI is not supported this way, you should run `transformers-cli env`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Do not Preload a Deep Learning Framework
Author: daskol
Pull request: no

### Feature request

I need some toggle (say envvar `HF_FRAMEWORK` or configuration option `transformers.config.framework`) or runtime check on importing `transformers` in order not to import `tensorflow`, `torch`, and `flax` simulteneously.

### Motivation

At the moment `transformers ` package imports all deep learning packages even if they are not used. In other words, if there are `tensorflow` and `torch` installed simultaneously then both packages will be imported despite the only `torch` is actually used.

### Your contribution

I'll see but I am not sure that it is easy to remove explicit imports of DL framework on `transformer` imports.

Comments:
Those environment variable exist, but Transformers also only imports the frameworks as needed, they are named `USE_TF`, `USE_TORCH` and `USE_JAX`.

Many thanks!

Are they described somewhere in documentation? I didn't manage to find them. Is it possible to add `HF_` prefix to be consistent with `HF_HOME` as an example?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fix MaskFormerImageProcessor.post_process_instance_segmentation
Author: alaradirik
Pull request: yes

# What does this PR do?
Fixes the `post_process_instance_segmentation` method of `MaskFormerImageProcessor`. This issue mainly affects Mask2Former as it uses MaskFormerImageProcessor and there aren't any MaskFormer models trained on instance segmentation datasets. 

Unlike panoptic segmentation post-processing, the final score of each binary mask proposal is calculated by multiplying the mask proposal score with the class score. `mask_threshold` and `overlap_mask_area_threshold` arguments are not needed anymore, I can either add a warning to deprecate them or leave it as it is for now.

Post-processed results of the `mask2former-swin-small-coco-instance` model inference:
![download](https://user-images.githubusercontent.com/8944735/214045581-d177f24b-a7c1-4ec6-9d12-453b8300eb92.png)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Thanks for fixing! Should we add a corresponding test for it, which verifies the postprocessed results?

I added a test but Mask2Former, unlike MaskFormer, outputs segmentation maps of shape (96, 96) instead of the preprocessed input size for efficiency. They scale the mask logits to the preprocessed image size during postprocessing (same for semantic and panoptic segmentation), even if no` target_sizes` is passed. I think it'd better to add an image processor for Mask2Former as its post-processing requires additional scaling.

What do you think @NielsRogge @sgugger?

If postprocessing is different, then it indeed requires its own image processor class.
Models not in eval()-mode when loaded with from_config()
Author: hadsed
Pull request: no

### System Info

The docs say that models loaded with `from_pretrained()` are done so with `model.eval()` mode on by default. But when using `from_config()` that's not the case, even though loading configs and tokenizers would be using `from_pretrained()` like so:
```python
config = AutoConfig.from_pretrained(
    MODEL_NAME,
    padding='max_length',
    truncation=True,
    output_hidden_states=True,
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, config=config)
model = AutoModelForSequenceClassification.from_config(config)
```
I'd like to argue that we should put the model in `eval()` mode when using `from_config()`. I know at least 2 other people who have spent a great number of hours validating and hunting for that. Similar reasoning to https://github.com/huggingface/transformers/issues/695#issuecomment-502964803 I think it's important to make things deterministic out of the box.

Or, open to understanding why that wouldn't be the case.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Try something like this:
```python
config = AutoConfig.from_pretrained(
    MODEL_NAME,
    padding='max_length',
    truncation=True,
    output_hidden_states=True,
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, config=config)
model = AutoModelForSequenceClassification.from_config(config)
```

### Expected behavior

I'd expect the model to be loaded in `eval()` mode.

Comments:
A model created with `from_config` will have random weights and is thus not suitable for inference. This is why it is put in training mode, as the documentation clearly states. In any case, it has been the case for such a long time that reverting this would surprise way more users with a breaking change.

The code snippet I posted above does not load random weights.

Yes it does.

Well that explains a lot. I stand corrected, thank you.
Issue Importing Image Resolution Models
Author: pravin-santhanam27
Pull request: no

Hey Everyone,

I am tying to import a model from transformers for deblurring images.

I am on Python 3.10.9 and just install transformers 4.25.1


The error comes on import

`from transformers import AutoImageProcessor, Swin2SRForImageSuperResolution`

and the error is:

`Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Swin2SRForImageSuperResolution' from 'transformers' (\env\lib\site-packages\transformers\__init__.py)`


These are the packages currently installed in my virtual environment

`certifi==2022.12.7
charset-normalizer==3.0.1
colorama==0.4.6
filelock==3.9.0
huggingface-hub==0.11.1
idna==3.4
numpy==1.24.1
opencv-python==4.7.0.68
packaging==23.0
Pillow==9.4.0
PyYAML==6.0
regex==2022.10.31
requests==2.28.2
tokenizers==0.13.2
torch==1.13.1+cu117
torchaudio==0.13.1+cu117
torchvision==0.14.1+cu117
tqdm==4.64.1
transformers==4.25.1
typing_extensions==4.4.0
urllib3==1.26.14`

Comments:
Hi, @pravin-santhanam27 there seems to be a problem loading `Swin2SRForImageSuperResolution` with stable transformers(4.25.1) which we install from pypi. But this error is not present if you install from the source(4.26.0.dev0). This error is fixed in the source version(which is regularly updated) and will also be updated to stable release later. If you want to use it right now then please install `transformers` from source - `pip install git+https://github.com/huggingface/transformers`

Will close this as the issue seems resolved.
[GenerationConfig] add additional kwargs handling
Author: ArthurZucker
Pull request: yes

# What does this PR do?
This add the same support that we have in the `PretrainedConfig`, where additional kwargs are automaticallu updated. 
This will allow users to re-use the `GenerationConfig` class for most of the use_cases, whithout having to add a model specific class. I was trying to load [the following `generation_config` ](https://huggingface.co/openai/whisper-small/discussions/10/files)and got half of my additional arguments deleted 😉  

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Also will have to add test + this is apparently breaking a lot of things haha 

Okay, after talking a bit with @gante and testing, this is not the best, this PR will focus on other missing functionalities. Mostly addition of the `dict_torch_dtype_to_str` function, as the `dtype` could be passed to the generation 😉 
The problem is mostly that if we process all the additional kwargs, we are getting all of the arguments from the `configuration.json` which mixes things up. 
The simplest solution is either to store them in `generate_kwargs` or re-write the configuration for the model. I though this was cumbersome but it is actually the most logical and cleanest way to do it. 

EDIT : gonna just add a condition, if the kwargs are from a config file, they are not added. 

Now only thing left is to add a pretty test with all the different edge cases I encountered.
DeBerta Wrong Dimension for MLM Prediction Head
Author: zanussbaum
Pull request: no

### System Info

From the [original implementation](https://github.com/microsoft/DeBERTa/blob/master/DeBERTa/deberta/mlm.py#L21), the MLM head transforms the hidden dimension to the embedding dimension. 

However, it seems that in the HF version, we go from `hidden_size` to `hidden_size`. Shouldn't it be from `hidden_size` to `embedding_size`, especially since the [embeddings get tied](https://github.com/huggingface/transformers/blob/v4.24.0/src/transformers/modeling_utils.py#L1203) eventually?

### Who can help?

@ArthurZucker and @younesbelkada

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

N/A

### Expected behavior

` DebertaPredictionHeadTransform.dense.weight` should be of size `hidden_size, embedding_size`, not `hidden_size, hidden_size`

Comments:
Not sure I understand where you found that we only have from `hidden_size` to `hidden_size`, but for `MLM`, we use the `DebertaOnlyMLMHead`, which uses a `decoder` that is the prediction head. See `self.decoder = nn.Linear(config.hidden_size, config.vocab_size, bias=False)` [here](https://github.com/huggingface/transformers/blob/main/src/transformers/models/deberta/modeling_deberta.py#L1146)

Ah sorry I forgot to include this line!
https://github.com/huggingface/transformers/blob/main/src/transformers/models/deberta_v2/modeling_deberta_v2.py#L872
It looks like it checks for an attribute `embedding_size` in the config, but defaults to `hidden_size` if not found. 

It doesn’t seem like that attribute is present in the config though? Am I setting up the model incorrectly?

Okay, again the head that you mentioned : `DebertaPredictionHeadTransform` is just a head, and it does not use the `DebertaV2Embedding`. It is only used as such (and not as an entire model). The size is correct as what you are looking for it in the `DebertaForMaskedLM`. The `self.model` attribute encompasses the `embedding` layer. 

Hm okay thanks for the help! Seems I misunderstood
[Whisper] fix all issues with unk token
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Previously, all OOV ( and thus timestamp tokens) outputed by the model are decoded to `
` by the `xxx.en` whisper models. This does not happen with the multilingual model only because I added `""` to the vocabulary, and the `unk_token_id` is the same `""`. But this does not really make sense. 
As the default behavior for Whisper is just to outptu `""` for any OOV, now the `_convert_id_to_token` function does not use a `unk_token`. 

This will fix the inconsistency, and will help for the whisper refactoring. 


Comments:
_The documentation is not available anymore as the PR was closed or merged._