noamaanMulla-03/transformers-issues huggingface dataset:
Finetune M2M on multiple language pairs
Author: nikhiljaiswal
Pull request: no

Hi @all, I am able to finetune the M2M model for one lang pair using the below script.

CUDA_VISIBLE_DEVICES=0,1,2,3,6 python -m torch.distributed.run --nproc_per_node=5 run_translation.py --model_name_or_path=m2m100_418M_new_token --do_train --do_eval --source_lang ja --target_lang en --fp16=True --evaluation_strategy epoch --output_dir bigfrall --per_device_train_batch_size=48 --per_device_eval_batch_size=48 --overwrite_output_dir --forced_bos_token “en” --train_file orig_manga/orig/train_exp_frame_50k.json --validation_file orig_manga/orig/valid_exp_frame_50k.json --tokenizer_name tokenizer_new_token --num_train_epochs 50 --save_total_limit=5 --save_strategy=epoch --load_best_model_at_end=True --predict_with_generate

But I do not find a solution to finetune the model on more than one language pair simultaneously which I could easily do using FairSeq.
E.g. now I want to finetune it on ja-en and ja-zh pairs. How to pass these both languages? I could not find an option in the run_translation.py script. Please help.

Comments:
hi, I don't think it's possible :( But you plan to make your model public?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

I also want to do this, do you solve it? @Jourdelune 

No :(

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add unpack_input decorator to ViT model
Author: johnnv1
Pull request: yes

# What does this PR do?
I added the unpack_input decorator for the ViT model as describe in #16051 

@gante 

Maybe can also delete the `**kwargs` from `call` parameters?  (tests seems unaffected)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> Maybe can also delete the **kwargs from call parameters? (tests seems unaffected)

That's a good point. Actually, I think this will be true for all models after this change. Do you mind if I add it to my backlog, to confirm after this sprint? If we were to change this, I would have to bring in other Hugging Face members in (changing the signature of a public method is sensitive).

STOP EMAILING ME!!!!!!!!!!!!!!!!!!!!!!

Sent from my iPhone

> On Mar 12, 2022, at 9:20 AM, Joao Gante ***@***.***> wrote:
> 
> ﻿
> @gante approved this pull request.
> 
> Fantastic, thank you so much for the contribution 👍
> 
> Can you confirm that you've run RUN_SLOW=1 py.test -vv tests/[model_name]/test_modeling_tf_[model_name].py? Will merge after your confirmation :)
> 
> —
> Reply to this email directly, view it on GitHub, or unsubscribe.
> Triage notifications on the go with GitHub Mobile for iOS or Android. 
> You are receiving this because you are subscribed to this thread.


Hi @debtriche 👋 GitHub doesn't send random emails, if you are receiving unwanted emails please check your subscription and emailing settings :)

> Fantastic, thank you so much for the contribution +1
> 
> Can you confirm that you've run RUN_SLOW=1 py.test -vv tests/[model_name]/test_modeling_tf_[model_name].py? Will merge after your confirmation :)

Yes, i run, but two test are skipped, all others pass
Add type hints to XLM model (PyTorch)
Author: jbrry
Pull request: yes

# What does this PR do?

This PR adds type hints for XLMModel (PyTorch), which is one of the listed models in #16059.
Type hints were added based on the example given in #16074.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?
@Rocketknight1 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Perceiver IO : How to preprocess raw audio into vector  
Author: hitesh159
Pull request: no

@NielsRogge  I am working with end to end application where I have to take raw video input and have to identify user sentiment from it . For that I want to know how to preprocess the raw audio to be used by PerceiverAudioPreprocessor. 

Comments:
I would really like to collaborate with @anton-l and @patrickvonplaten to make an audio example for Perceiver IO.

Basically, the `PerceiverAudioPreprocessor` turns a batch of raw audio streams into a tensor of shape `(batch_size, seq_len, hidden_size)`. This is often called `hidden_states` in Transformers (it's like the initial tensor you provide to a Transformer encoder). 

The way you encode your audio to these hidden states doesn't matter, there are multiple ways.

* `PerceiverAudioPreprocessor` is one example. It just takes the raw audio and adds position embeddings (trainable/Fourier) to it.
* Another example is using Wav2Vec2's method, as follows:

```
from transformers import Wav2Vec2FeatureExtractor
from transformers.models.wav2vec2.modeling_wav2vec2 import Wav2VecFeatureWav2Vec2FeatureEncoder
from datasets import load_dataset

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
sampling_rate = dataset.features["audio"].sampling_rate

feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("facebook/wav2vec2-base-960h")
config = Wav2Vec2Config.from_pretrained("facebook/wav2vec2-base-960h")
feature_encoder = Wav2Vec2FeatureEncoder(config)

# construct raw audio
# audio file is decoded on the fly
inputs = feature_extractor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")

# turn raw audio into hidden_states of shape (B, L, C)
hidden_states = feature_encoder(**inputs)
```

However, in this case, there are no position embeddings added it seems (not sure if Wav2Vec2 uses position embeddings).

Next, you can do cross-attention between the `latents` of Perceiver, and the `hidden_states` we just created.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Add missing type hints for all flavors of LayoutLMv2 PyTorch models.
Author: chainyo
Pull request: yes

# What does this PR do?

I added type hints for all `LayoutLMv2` PyTorch classes as described in https://github.com/huggingface/transformers/issues/16059 .

@Rocketknight1 



Comments:
This also looks good, but same comment about the return types!

_The documentation is not available anymore as the PR was closed or merged._

> This also looks good, but same comment about the return types!

Yes I fixed return types + I added all type hints for `LayoutLM` PyTorch 

Hi, I'm seeing some tests failing on this one - I think you might have deleted a couple of arguments while adding the annotations!
Add TFCamembertForCausalLM and ONNX integration test
Author: lewtun
Pull request: yes

# What does this PR do?

This PR adds a missing causal language modelling head for the TensorFlow implementation of Camembert. 

With this, we can also test the ONNX export of TensorFlow Camembert models, so I've included the checkpoint in the integration tests as well

This is my first TensorFlow contribution to `transformers`, so please be nice :)

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix and document Zero Shot Image Classification
Author: osanseviero
Pull request: yes

# What does this PR do?

This PR fixes 3 issues with `zero-shot-image-classification`

* It does not pick a default model, although there is one. This leads to weird error message

```python
pipe = pipeline("zero-shot-image-classification")
>> ValueError: The task defaults can't be correctly selected. You probably meant "translation_XX_to_YY"
```

* The pipeline is not in the list at top of https://huggingface.co/docs/transformers/main_classes/pipelines
* The filter for available models has a typo, although right now this still leads to no models (ok just one).

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@Narsil @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Change unpacking of TF mpnet, rag, roformer inputs to use decorator
Author: vumichien
Pull request: yes

# What does this PR do?

As part of #16051 this PR changes the unpacking of the inputs of TF mpnet, rag, roformer to use the decorator instead.

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@Rocketknight1  @gante 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

These changes are also in #16112 -- can I close this PR? :)

Sorry about this mess.  I will close it

> Sorry about this mess. I will close it

No worries! Thank you for closing the PR
Fix ProphetNetTokenizer
Author: ydshieh
Pull request: yes

# What does this PR do?


Fix `ProphetNetTokenizer`: `ProphetNet` doesn't have `token_type_ids` as argument.

Without this PR, the docstring examples will give the following errors:

```
in _call_impl
                return forward_call(*input, **kwargs)
            TypeError: forward() got an unexpected keyword argument 'token_type_ids'
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Change unpacking of TF mobilebert inputs to use decorator
Author: vumichien
Pull request: yes

# What does this PR do?

As part of #16051 this PR changes the unpacking of the inputs of TF mobilebert to use the decorator instead.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?

## Who can review?

@Rocketknight1 @gante 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

@gante I have run the test and all pass. But when I run `make fixup` in my windows laptop it throws the error `-n was unexpected at this time.
make: *** [Makefile:10: modified_only_fixup] Error 255
` Do you know how to fix it?

> @gante I have run the test and all pass. But when I run `make fixup` in my windows laptop it throws the error `-n was unexpected at this time. make: *** [Makefile:10: modified_only_fixup] Error 255 ` Do you know how to fix it?

Check our guide [here](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#develop-on-windows) :)

Thank you :D
Steps strategy fix for PushtoHubCallback and changed docstring
Author: merveenoyan
Pull request: yes

Hello 👋 
Steps strategy for `PushtoHubCallback` wasn't working because

`if self.save_strategy == IntervalStrategy.STEPS and batch + 1 % self.save_steps == 0:`
this is only true when `self.save_steps` is 1 and batch is 0 (due to `1 % self.save_steps` preceding +) So I observed a weird behavior where it was only pushing at 0 batch when save steps were 1 and it wasn't pushing when save steps weren't 1.

This is a small fix that does 
`if self.save_strategy == IntervalStrategy.STEPS and (batch + 1) % self.save_steps == 0:` which fixes the callback.

I also changed the description for `no` in docstring.

I will also add tests for the callback later.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[ViTMAE] Add copied from statements and fix prefix
Author: NielsRogge
Pull request: yes

# What does this PR do?

Fixes #16101 by updating `base_model_prefix` from "vit_mae" to "vit" (`ViTMAEForPreTraining` is using the latter).

Also adds copied from statements to ensure consistency with `modeling_vit.py`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Spanish translation of the file training.mdx
Author: yharyarias
Pull request: yes

I made the translation of the **training.mdx** file in collaboration to the transformers/doc/ documentation, there I created a new folder called source_es where I hosted my training.mdx document translated to Spanish, contributing to the Spanish-speaking community. 
[forum](https://github.com/huggingface/transformers/issues/15947)

Approved by @omarespejel
Documentation: @sgugger




Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger Of course, I will make the changes you indicate, leaving the syntax as per the original documentation. I have a question, I see that most of the conflicts are because of the links, do I leave all the links or remove them? I think these are important.

As soon as the syntax is reviewed by @yharyarias, I will review the Spanish.

The links are automatically generated by our doc building tool. You should leave them as they are in the `training.mdx` file English version (for instance [`Trainer`]).

Perfect! I'll follow your suggestions, thank you @sgugger 🙌🤗

I just uploaded the changes you suggested @sgugger I'll be watching for your review.

Thanks! Will review the translation.

I made the last changes suggested by @omarespejel , I think everything is ok, this is my first contribution and there will be more to come, I have loved collaborating with open source. Thank you very much 🤗 @sgugger @LysandreJik  and @omarespejel 

Thanks, @yharyarias! There are still a couple of comments unresolved. 

IMO it is good to go :+1: Thank you @yharyarias!  

Thanks again for your PR. Merging this but we will wait a little bit more to serve a doc in Spanish mainly:
- to have a few more pages
- to automatically redirect to English doc when the page is not available in Spanish.

I will put a comment here when the page is live (hopefully in a week or two!)

@sgugger Thank you, of course I will continue to contribute, I'm currently working on the translation of another document to add to source_es.
[Fix doc example] Fix checkpoint name in docstring example
Author: ydshieh
Pull request: yes

# What does this PR do?

Use real name instead of `_CHECKPOINT_FOR_DOC` in docstring example.

## More context

This line in `modeling_speech_to_text_2.py`
```python
>>> tokenizer = Speech2Text2Tokenizer.from_pretrained(_CHECKPOINT_FOR_DOC)
```

gives the following error:
```python
Traceback (most recent call last):
  File "/tmp/tmpgv1pik63/-789619136090462200.py", line 13, in <module>
    tokenizer = Speech2Text2Tokenizer.from_pretrained(_CHECKPOINT_FOR_DOC)
NameError: name '_CHECKPOINT_FOR_DOC' is not defined
```

because `_CHECKPOINT_FOR_DOC` is not defined if a user copies the code example and try to run it.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hey @ydshieh,

Could you explain this change in a bit more detail? 

> Hey @ydshieh,
> 
> Could you explain this change in a bit more detail?

A docstring example uses the variable _CHECKPOINT_FOR_DOC as the argument for the method from_pretrained. But for a user who copies the example and tries to run it, an error occurs because _CHECKPOINT_FOR_DOC is not defined. It is only defined inside the model file.

I think the code example should be self-contained, and could run after copy-paste.

(I also updated the PR description)
Better input variable naming for OpenAI (TF)
Author: bhavika
Pull request: yes

# What does this PR do?

Uses the @unpack_inputs decorator instead of input_processing for OpenAI (TF). See https://github.com/huggingface/transformers/issues/16051

Tagging @Rocketknight1 for review! 

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._

I've also made the same edit on the PR description :)

@gante Made changes as you suggested and I'm confirming that the tests pass for `RUN_SLOW=1 py.test -vv tests/openai/test_modeling_tf_openai.py`
Add type hints for TFDistilBert
Author: PepijnBoers
Pull request: yes

This PR adds type hints for TFDistilBert, following the example of https://github.com/huggingface/transformers/pull/16057 as mentioned by @Rocketknight1 in issue https://github.com/huggingface/transformers/issues/16059.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
FeaturesManager assumes only one of Torch or TensorFlow is installed
Author: ndronen
Pull request: no

## Environment info

- `transformers` version: 4.12.5
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.9.10
- PyTorch version (GPU?): 1.10.0 (False)
- Tensorflow version (GPU?): 2.8.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

### Who can help

@michaelbenayoun @Albertobegue 

## Information

When both Torch and TensorFlow are installed, `FeaturesManager` defaults to using `AutoModel`, so the model returned by `get_model_from_feature` is always Torch.

## To reproduce

Steps to reproduce the behavior:

1. Install Torch and TF
2. Call `FeatureManager.get_model_from_feature` with arbitrary but supported `features` and `model_name` arguments
3. The resulting model is always a Torch model

```python
features = "default" # randomly chosen, supported feature
model_name = "bert" # randomly chosen, supported model
model = FeaturesManager.get_model_from_feature(features, model_name)
```

## Expected behavior

Some test environments have both Torch and TensorFlow installed, because the immediate task is to ensure functionality is the same regardless of the framework. I would expect `FeaturesManager.get_model_from_feature` to allow TensorFlow to be used even when Torch is installed. This could be implemented by e.g. a keyword argument to `get_model_from_feature` with a default value of `None`. When the keyword argument is `None`, and both Torch and TensorFlow are installed, `FeatureManager` would default to Torch, as it does now. Otherwise, it would use the specified framework.

Comments:
Also cc @lewtun 

I'm on it!

> I'm on it!

Thanks!