noamaanMulla-03/transformers-issues huggingface dataset:
wrong wav2vec link in audio classification blog
Author: chrisemezue
Pull request: no

[This wonderful blog](https://huggingface.co/docs/transformers/main/en/tasks/audio_classification#preprocess) has an issue with the wav2vec model card link. See below:  


> 2. Check the sampling rate of the audio file matches the sampling rate of the audio data a model was pretrained with. You can find this information on the Wav2Vec2 `[model card]((https://huggingface.co/facebook/wav2vec2-base))`.

The right link should be https://huggingface.co/facebook/wav2vec2-base



Comments:
Closed by #18945
MaskFormerFeatureExtractor doesn't process instance segmentation maps correctly
Author: alaradirik
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.0.dev0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.8.13
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.11.0 (False)
- Tensorflow version (GPU?): 2.9.1 (False)
- Flax version (CPU?/GPU?/TPU?): 0.5.0 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help?

@NielsRogge @sgugger 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`MaskFormerFeatureExtractor` takes the following arguments as input:
- images: images to be segmented
- segmentation_maps (optional): can either be pixel-wise class annotations (default) or pixel-wise instance id annotations
- instance_id_to_semantic_id (optional): a dictionary (`Dict[int, int]`) that maps instance ids to class ids. If this is given as input, `segmentation_maps` inputs are treated as instance segmentation maps

Configuration arguments:
- reduce_labels (optional): decrements segmentation map values by 1, should be set to `True` if the dataset labels start from 1
- ignore_index (optional): `background` pixel values denoted with 0 are replaced with the `ignore_index `

If `instance_id_to_semantic_id` is provided, `MaskFormerFeatureExtractor` needs to create binary masks for each object instance in the image and should be able to handle overlapping objects of the same category. The binary masks then should be mapped to their corresponding class id. However, the current implementation of `convert_segmentation_map_to_binary_masks()`:
- Performs label reduction before mapping instance IDs to class IDs
- Converts instance segmentation maps to semantic segmentation masks before creating binary masks, causing the instance level information to be lost

### Expected behavior

If instance segmentation maps are provided as `segmentation_maps` to `MaskFormerFeatureExtractor.convert_segmentation_map_to_binary_masks()`:

1. segmentation_maps should be directly used to create binary masks
2. `instance_id_to_semantic_id` mapping should be used to map binary mask values (insance IDs) to corresponding object class IDs 

Comments:
Fix MaskFormerFeatureExtractor instance segmentation preprocessing bug
Author: alaradirik
Pull request: yes

# What does this PR do?
- Updates `MaskFormerFeatureExtractor` docstrings for clarity
- Fixes bug in `MaskFormerFeatureExtractor` that causes instance segmentation maps to be processed incorrectly
- Adds support to use per-image instance_id_2_semantic_id mappings

Fixes #18989


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Generate: add model class validation
Author: gante
Pull request: yes

# What does this PR do?

Fixes #18210 

This PR adds model class validation at the start of generate (all model classes inherit `GenerationMixin`, but few can use `generate()`). It also adds an exception that attempts to redirect the users to the right classes.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger @patrickvonplaten I've requested a re-review of this PR. As per @patrickvonplaten's suggestion, the PR was upgraded to contain the exact class the user should use in the exception (as opposed to pointing to all generate-compatible auto classes).

In the process of building it, I've noticed that the previous version of this PR was incorrect anyways -- PT and TF had a default `prepare_inputs_for_generation`, so we couldn't rely on its existence. Only 1 model was using this default, so I removed it and implemented it in the missing model. The default `prepare_inputs_for_generation` was a public method, but since this PR blocks the use of `generate()` with classes that are not intended to be used with it anyways, removing the public method should have little impact. Nevertheless, it is a point to consider in the review!

_______________________________

Here's an example with the current version of the PR:
```
>>> from transformers import AutoModel
>>> model = AutoModel.from_pretrained("distilgpt2")
>>> model.generate("foo")
TypeError: The current model class (GPT2Model) is not compatible with `.generate()`, as it doesn't have a language model head. Please use one of the following classes instead: {'GPT2LMHeadModel'}
```
Fix tokenizer class for `XLMRobertaXL`
Author: ydshieh
Pull request: yes

# What does this PR do?

The 2 checkpoints for `XLMRobertaXL` use `XLMRobertaTokenizer` as `tokenizer_class`. In my PR #16857, I only checked `_CONFIG_FOR_DOC` in the modeling file at that time, and took `RobertaTokenizer` from there, which was wrong.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Update default revision for document-question-answering
Author: ankrgyl
Pull request: yes

# What does this PR do?

Prior to this change, users needed to instantiate a tokenizer themselves while using `impira/layoutlm-document-qa` to set the `add_prefix_space=True` parameter. I made this the default in the tokenizer's config [here](https://huggingface.co/impira/layoutlm-document-qa/commit/52e01b37ccf248953eb527c1d96e9ec1750f3c3c), and this change simply updates the pinned revision to reference it.

After this change, the following commands work:

```
In [1]: from transformers import AutoTokenizer, pipeline
In [2]: nlp = pipeline('document-question-answering')
In [3]: nlp(
   ...:     "https://templates.invoicehome.com/invoice-template-us-neat-750px.png",
   ...:     "What is the invoice number?"
   ...: )
Out[3]: {'score': 0.9998127222061157, 'answer': 'us-001', 'start': 15, 'end': 15}
```

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@NielsRogge 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Gentle nudge @Narsil @NielsRogge
mypy typing not working for AutoModelForMaskedLM when used with Trainer
Author: harshit-sethi09
Pull request: no
Labels: name: bug

### System Info

Python version - 3.9
transformers version - 4.20.1

### Who can help?

@sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Initialise `AutoModelForMaskedLM` 
model = `AutoModelForMaskedLM.from_pretrained("xlm-roberta-base")`

2. Pass this to the Trainer
```
trainer = Trainer(
            model= model,
            args=training_config,
            train_dataset=train_data,
            eval_dataset=valid_data,
            callbacks=None,
            data_collator=masking_processor
        )
```

3. If you check this against mypy, it produces an error stating 
`error: Argument "model" to "Trainer" has incompatible type "AutoModelForMaskedLM"; expected "Union[PreTrainedModel, Module]"`

The model is defined as `model: Union[PreTrainedModel, nn.Module] = None` in the Trainer class.

### Expected behavior

Since it's a valid input to the Trainer class, it should expect the models from AutoModel classes as well.

Comments:
Hi there @harshit-sethi09 👋 Our code has type annotations, but they are mostly for documentation purposes (and not to be used with `mypy`)

cc @sgugger 

I have very strong thoughts about trying to statically type-checking a dynamically typed language which would take too long to express here. But this is what you get as a result: an object will actually never be of type `AutoModel` has those can only be instantiated via classmethods which actually return other classes, a thing the static type-checker is of course incapable to see. 

We could pollute the code with tons of useless annotations to please the almighty static typechecker but we have chosen not to :-)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Multi-GPU setting: Expected to mark a variable ready only once (RuntimeError)
Author: sajastu
Pull request: no
Labels: name: bug

### System Info

I’m trying to run a Huggingface model on a multi-GPU environment. The problem is that when I’m processing multiple inputs which are bound to each other from a single class (shared-weights encoder), I’m getting `RuntimeError: Expected to mark a variable ready only once`. While if I use this encoder module only once, for processing the first input, I won’t get this error, in multi-GPU setting. Should add that I don't get this error when training on single GPU. 

To make it clearer, here is the structure (I have simplified the code):

```
class LEDModel():
    
    def __init__(self, ...)
	   self.encoder = ...


    def forward(input_ids, ...):

	   encoder_outputs = self.encoder(input_ids, ...)

	   # filter encoder_outputs and construct another tensor called 'input_ids_selected'

	   encoder_outputs = self.encoder(input_ids_selected, ...)
           ...

	   return LEDSeq2SeqModelOutput(
            last_hidden_state=decoder_outputs.last_hidden_state,
            past_key_values=decoder_outputs.past_key_values,
            sent_scores=sent_scores,
            sect_scores=sect_scores,
            decoder_hidden_states=decoder_outputs.hidden_states,
            encoder_last_hidden_state=encoder_outputs.last_hidden_state,
        )
```

when I remove this line: `encoder_outputs = self.encoder(input_ids_selected, ...)`, I don't run into this error. Should say that to filter `encoder_outputs` from the first pass of the encoder, I’m using other modules (linear layers) to find important `input_ids`, retaining those in input_ids_selected. You can perceive this as a two-step summarizer.



### Who can help?

@patrickvonplaten @ydshieh @patil-suraj

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Having a weight-shared module inside LEDModel() class will reproduce this error. 

### Expected behavior

Currently, I'm getting `Expected to mark a variable ready only once (RuntimeError)` **in multi-GPU** configuration. I expect to run this model flawlessly in this setting. 

Comments:
Hi @sajastu Could you share a (minimal) code snippet that could reproduce this issue? Thank you.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Pre-processing re-runs for each process
Author: rahular
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.22.0.dev0
- Platform: Linux-4.15.0-180-generic-x86_64-with-glibc2.27
- Python version: 3.10.4
- Huggingface_hub version: 0.8.1
- PyTorch version (GPU?): 1.12.1+cu102 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

@muellerzr @sgugger @patil-suraj 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In `examples/summarization_no_trainer.py`, we use the following code to pre-process the data:
```
    with accelerator.main_process_first():
        processed_datasets = raw_datasets.map(
            preprocess_function,
            batched=True,
            num_proc=args.preprocessing_num_workers,
            remove_columns=column_names,
            load_from_cache_file=not args.overwrite_cache,
            desc="Running tokenizer on dataset",
        )
```
In a multi-GPU (process) setup, the main process pre-processes the data first and saves it in the cache. Ideally, the other processes should just pick it up from there. But that's not happening. Every process is re-pre-processing the data. This is a problem when the data is large.

I have tried to check if something changes during the run with `Hasher.hash(preprocess_func)`. But the hash remains the same.

### Expected behavior

Processes other than the main process should read the processed data from cache.the 

Comments:
Do you have a reproducer of the behavior you are seeing? On my side data is processed once.

Wow, ok. So looks like the example scripts always set `--overwrite_cache` to `True`.

It is currently `parser.add_argument("--overwrite_cache", type=bool, default=None)` instead of `parser.add_argument("--overwrite_cache", action="store_true")`.

Will make a PR soon.
Fixed bug which caused overwrite_cache to always be True
Author: rahular
Pull request: yes

Many example scripts currently do `parser.add_argument("--overwrite_cache", type=bool, default=None)` which always sets the argument to `True` no matter what value is passed.

Fixes #18967

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger this fixes #18967. Please review.
add missing `require_tf` for `TFOPTGenerationTest` 
Author: ydshieh
Pull request: yes

# What does this PR do?

On scheduled CI, it was fine, as the docker image have TF installed.
On past CI project, it failed due to the lack of TF.

In any case, we should have `require_tf` for this test.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Re-add support for single url files in objects download
Author: sgugger
Pull request: yes

# What does this PR do?

During the cache revamp done in #18438, we accidentally lost support for single urls in `from_pretrained` methods (something like `config = AutoConfig.from_pretrained("http://my_custom_config.json")`. This is not something we want to support in the long run as users should use the Hub to store their objects, but this is still a breaking change.

This PR adds support for this corner case with proper deprecation warnings. Note that such urls are not cached anymore.

Comments:
cc @alaradirik 

_The documentation is not available anymore as the PR was closed or merged._
Fix typo in add_new_model_like.py
Author: mathemakitten
Pull request: yes

(feel free to ignore; trying to see if this triggers CI successfully as it's not working for me on other HF repos)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @mathemakitten, let's try to solve the CircleCI issue! Could you try refreshing your permissions as shown [here](https://support.circleci.com/hc/en-us/articles/360048210711-How-to-Refresh-User-Permissions-)?

Hi @LysandreJik, thanks for the link — I refreshed the user permissions in the CircleCI web interface and tests seem to trigger as expected now for my PRs!

Glad to hear it! Would you mind pushing a new commit (empty or not) to this branch so that it runs the check here and we can merge?
Remove unused `activation_dropout` in OPT
Author: shijie-wu
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/issues/18309

PR for corresponding models
* https://huggingface.co/facebook/opt-125m/discussions/19
* https://huggingface.co/facebook/opt-350m/discussions/5
* https://huggingface.co/facebook/opt-1.3b/discussions/7
* https://huggingface.co/facebook/opt-2.7b/discussions/5
* https://huggingface.co/facebook/opt-6.7b/discussions/8
* https://huggingface.co/facebook/opt-13b/discussions/7
* https://huggingface.co/facebook/opt-30b/discussions/8
* https://huggingface.co/facebook/opt-66b/discussions/7


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@ArthurZucker 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ArthurZucker I am not sure who is the owner of the models repo but the following PRs also need to be merged.

> 
> https://huggingface.co/facebook/opt-125m/discussions/19
> https://huggingface.co/facebook/opt-350m/discussions/5
> https://huggingface.co/facebook/opt-1.3b/discussions/7
> https://huggingface.co/facebook/opt-2.7b/discussions/5
> https://huggingface.co/facebook/opt-6.7b/discussions/8
> https://huggingface.co/facebook/opt-13b/discussions/7
> https://huggingface.co/facebook/opt-30b/discussions/8
> https://huggingface.co/facebook/opt-66b/discussions/7
> 

Hey, thanks for notifying, I will take care of it 😃
Add Support to Gradient Checkpointing for LongT5
Author: agemagician
Pull request: yes

# What does this PR do?

FlaxLongT5PreTrainedModel is missing "enable_gradient_checkpointing" function. This gives an error if someone tries to enable gradient checkpointing for longt5:

```
    model.enable_gradient_checkpointing()                                                                                                                                                                                       File "/...../transformers/src/transformers/modeling_flax_utils.py", line 233, in enable_gradient_checkpointing
    raise NotImplementedError(f"gradient checkpointing method has to be implemented for {self}")
NotImplementedError: gradient checkpointing method has to be implemented for <transformers.models.longt5.modeling_flax_longt5.FlaxLongT5ForConditionalGeneration object at 0x7fa158153040>
```

This pull request fixes it.

## Before submitting
- [] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sanchit-gandhi could you take a quick look here?
Make AutoProcessor a magic loading class for all modalities
Author: sgugger
Pull request: yes

# What does this PR do?

This PR re-enables a feature initially part of #14465 : the fact that `AutoProcessor` is a class loading the right processing class for any model (so processor, tokenizer or feature extractor). You can thus do:
```
processor = AutoProcessor.from_pretrained("bert-base-cased")
# Returns a fast BERT tokenizer
```
or
```
processor = AutoProcessor.from_pretrained("facebook/convnext-tiny-224")
# Returns a fConvNext feature extractor
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._