noamaanMulla-03/transformers-issues huggingface dataset:
Problems loading csv dataset in examples `run_summarization.py`
Author: PhilipMay
Pull request: no

Hi,

the following lines in `run_summarization.py` have some issues.

https://github.com/huggingface/transformers/blob/2c2a31ffbcfe03339b1721348781aac4fc05bc5e/examples/pytorch/summarization/run_summarization.py#L344-L346

According to the datasets documentation a csv dataset is loaded by providing `data_files`. See https://huggingface.co/docs/datasets/loading.html#csv

This is not done here. AFAIK the code is not able to load a local dataset at all.

Furthermore, the dataset is always automatically loaded as a "train" dataset (which is default). 
So it is not possible to specify a validation or test dataset.

Comments:
Hi @PhilipMay 

This part of code https://github.com/huggingface/transformers/blob/a63bd3675f3fa1a6154c8bf1d085c66eaea67e56/examples/pytorch/summarization/run_summarization.py#L347-L358

handles loading `csv` or `json` files. It also handles train and valid and test splits. Have you tried passing the `train_file` , `validation_file` options to the script ? 

Yes. That is right. I have confused the two if branches. Sorry.
Electra model class support loading weights from other types of BERTs
Author: congchan
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

Electra model class support loading weights from other types of BERTs(roberta, bert, etc..).
Most of them share the same backbone, so is it possible to support this features? 

Thanks.

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->


Comments:
Hello, thanks for opening an issue and feature request! Could you enlighten me as to why you'd like that feature? 

I haven't thought deeply about it, but you're right that these models do share a very similar architecture! As we try to have our tools be as model agnostic as possible, I'm wondering what purpose loading BERT's weight in ELECTRA would serve, vs. simply loading BERT's weights in BERT?

Hi, my purpose is that I need a "post-train" phase between "pre-train" and "fine-tune" for downstream domain adaptation. Due to limited pre-trained resource, I have to adapt a pre-train model with faster electra training method to transfer an excellent pre-train models to downstream domain, so that I could fine-tuning with different tasks, or projects.


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
[Community Event] Robust Speech Challenge
Author: patrickvonplaten
Pull request: no

Hey everybody,

We have organized a new speech recognition community event and would love to have you there! 

During the event we want to teach you how to build robust speech recognition systems in your favorite language 🌍🌎🌏.
The event will focus on [XLS-R](https://huggingface.co/facebook/wav2vec2-xls-r-1b), [N-gram boosted Decoding](https://huggingface.co/blog/wav2vec2-with-ngram), and [Common Voice 7,8](https://commonvoice.mozilla.org/en/datasets).

For more information and to join the event, please take a look at this [**forum post**](https://discuss.huggingface.co/t/open-to-the-community-robust-speech-recognition-challenge/13614).

Open Sourcely,
the HuggingFace team! 🤗

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
add VisionTextDualEncoder and CLIP fine-tuning script
Author: patil-suraj
Pull request: yes

# What does this PR do?

This PR adds a fine-tuning script for `VisionTextDualEncoder` and `CLIP` model.  

The script is named `run_clip.py` since clip also refers to the training scheme (contrastive-language-image) and not just the model.  But feel free to suggest other names.


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Self-Attention Layers for Perceiver Decoder
Author: VitorGuizilini-TRI
Pull request: no

This feature request is about the PerceiverIO codebase, specifically about the implemented decoders. It seems like there are no self-attention layers, only a single cross-attention layer.  Is that by design?  Several current published methods mention cross- and self-attention layers as part of the decoding process, would it be possible to include this functionality?   

https://github.com/huggingface/transformers/blob/7799b6128feb17b63c47ed77a71fa367d26492d2/src/transformers/models/perceiver/modeling_perceiver.py#L2035

Comments:
Hi,

> It seems like there are no self-attention layers, only a single cross-attention layer. Is that by design?

Yes, the authors used only a single cross-attention layer. 

If you want, you can fork the library and tweak `modeling_perceiver.py` to include additional self-attention layers in the decoder.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Closing this issue, as I believe I've answered your question. Feel free to re-open if you have further questions.
Text generation with TAPAS as encoder
Author: perlitz
Pull request: no

Hi @NielsRogge,
Thanks for the TAPAS implementation!

I'm trying to to train the model to perform text generation conditioned on tables.
Since TAPAS can encode the semi-structured meaning in tables, I guessed it was a good choice to use it as an encoder and say GPT2 as a decoder.

I however encountered a problem when trying to generate from that EncoderDecoder model, this:
![image](https://user-images.githubusercontent.com/4630195/149194029-d7a664b6-374d-4dc9-a2dd-fb3bf007cd93.png)

results in the following error:
![image](https://user-images.githubusercontent.com/4630195/149193959-601a3129-7bc8-49ad-aa72-2463158f7202.png)

I guess this is since model.generate() for EncoderDecoder does not expect to have the extra token_type_ids that TAPAS has. Can you think of a way I can make this work?

Thanks!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

This question has been answered on the [forum](https://discuss.huggingface.co/t/using-generate-with-tapas-as-encoder-in-encoderdecoder/13655).
Add layer_idx to CrossAttention of GPT2 model
Author: hyunwoongko
Pull request: yes

When I tried `EncoderDecoderModel` with GPT2 model + `_upcast_and_reordered_attn`, cross attention layer did not have `layer_idx`, so the following error occured.

```
t2.py", line 240, in _upcast_and_reordered_attn
    scale_factor /= float(self.layer_idx + 1)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'
```

cc @patrickvonplaten @LysandreJik


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Looks good, thanks!

Once being approved by patrickvonplaten/LysandreJik/sgugger , could you also apply the same change to `modeling_imagegpt.py`, please:

https://github.com/huggingface/transformers/blob/2c2a31ffbcfe03339b1721348781aac4fc05bc5e/src/transformers/models/imagegpt/modeling_imagegpt.py#L426

@ydshieh Sure. 

Thanks!
TF text classification examples
Author: gante
Pull request: yes

# What does this PR do?

This PR updates the text classification examples to use the more modern `to_tf_dataset()` method, as opposed to a custom function. A few minor fixes were added as I navigated related files.

Example of running command: `python run_glue.py --model_name_or_path distilbert-base-cased --task_name mnli --do_train --do_eval --output_dir $HOME/test_model`

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Trainer can't estimate SpeechEncoderDecoder tokens for `floating_point_ops`
Author: OllieBroadhurst
Pull request: no

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.16.2
- Platform: Ubuntu
- Python version: 3.7
- PyTorch version (GPU?): 1.9
- Tensorflow version (GPU?): NA
- Using GPU in script?: Y
- Using distributed or parallel set-up in script?: N

### Who can help
Maybe @sgugger?

## Information

Model I am using (Bert, XLNet ...): SpeechEncoderDecoder

Training the model results in `Could not estimate the number of tokens of the input, floating-point operations will not be computed` warnings due to [this function](https://github.com/huggingface/transformers/blob/240cc6cbdc07fc1a7c103d71cfe38d5e60e489e4/src/transformers/modeling_utils.py#L378) being unable to find the correct `self.main_input_name`. For `SpeechEncoderDecoderModel`s, the default name is [`inputs` ](https://github.com/huggingface/transformers/blob/v4.16.2/src/transformers/models/speech_encoder_decoder/modeling_speech_encoder_decoder.py#L183) even though the encoder should either accept `input_values` or `input_features`. The function expects `inputs` to be in the `input_dict` but could only find the main input name of the encoder.

## To reproduce

Steps to reproduce the behavior:

1. Train a `SpeechEncoderDecoderModel` using the `Seq2SeqTrainer` and monitor console output.

## Expected behavior

The function should be able to determine the correct `main_input_name` of a `SpeechEncoderDecoderModel`. Presumably that of the encoder. I'm not sure if this applies to `VisionEncoderDecoderModel`s but I suspect it would.


Comments:
I don't think this can be solved since the model input name is not always the same. You can just ignore the warning.

cc @patrickvonplaten for the model input name, to see if there is a way to fix it.

@OllieBroadhurst, 

I think we could solve this if we rename `input_values` to `inputs` in the speech-encoder-decoder training script here: https://github.com/huggingface/transformers/blob/2f2fefd6afa496ea72312ff38686706717f8aa22/examples/pytorch/speech-recognition/run_speech_recognition_seq2seq.py#L197 

Could you try this out? Think this is the cleanest way of solving it

cc @sanchit-gandhi 

Nice idea @patrickvonplaten! I have my own collate function anyway so it's a simple change. Of course you'll be changing `input_features` to `inputs` if you're using `Speech2Text` (as opposed to `input_values`).
added link to our writing-doc document
Author: FrancescoSaverioZuppichini
Pull request: yes

# What does this PR do?

This PR adds a link to our writing-documentation guide in the add new model guide.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


Comments:
_The documentation is not available anymore as the PR was closed or merged._
TF train_step docstring
Author: gante
Pull request: yes

# What does this PR do?

Small PR that elaborates on the beautiful black magic behind our TF `train_step`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

LGTM! Maybe wait until tomorrow to merge because people are off, though, and I want to confirm a verbose docstring like this is okay.

Tagging @sgugger to check @Rocketknight1's question above ☝️ 

Also, don't forget `make style`!

> Can we break the docstring in several paragraphs? For instance at the "However" and then "This is" (but whatever you think is best works too).

Done 👍
Gelu10
Author: mfuntowicz
Pull request: yes

Introduce `GeLU10(x)` activation which behaves exactly like a raw GeLU but keeps output values within [-10, 10] range.

This is especially useful when quantizing neural network using GeLU activations, because it allows to map 2 negatives values within the GeLU spectrum.

Please see https://arxiv.org/abs/2004.09602 (Appendices D  "Novel activation functions"): 
> GELU has an output range of [−0.1700, ∞]. This
poses a challenge for uniform quantization as it should represent both small negative values and large positive values [...] However, if we restrict the range to [-10,10] then two negative values can be represented.

![image](https://user-images.githubusercontent.com/2241520/154236552-9ea2aaa7-d14b-4b14-924c-2bd4218c6243.png)


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Can this be used right now with some existing pretrained models?

@LysandreJik, yes it can be used anywhere there is a `gelu` call. 

The purpose of this operator is mainly to be used when doing the calibration phase for quantization, to avoid having to map the whole GeLU's range.

Other than this, not so much interest. 

At first, I was looking at putting this into optimum, but `transformers` has all the logic to handle this quite easily, wdyt?

It will need to be rebased/adapted to include the class the new activation function.
Plus the docstrings should be in our format ;-)

Sure, will go through it over the weekend 👌🏻 

@sgugger If you can validate regarding the doc? 🙏🏻
Fix typo on examples/pytorch/question-answering
Author: dreamgonfly
Pull request: yes

# What does this PR do?
Fix typo on examples/pytorch/question-answering/README.md (cna -> can).

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
Cleanup transformers-cli
Author: julien-c
Pull request: yes

Note to maintainers:
- the `ListReposObjsCommand` (using `HfApi.list_repos_objs`) (which has been deprecated for a while) is going to be removed server-side soon (cc @coyotte508 @XciD @Pierrci) so we should also remove them from transformers now IMO
- the other stuff is just removing deprecation errors (no actual functionality is removed) so I'd advocate to also remove it now. (no need to wait for a major version IMHO)

Related PR: https://github.com/huggingface/huggingface_hub/pull/702

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix `HfArgumentParser` when passing a generator
Author: bryant1410
Pull request: yes

# What does this PR do?

It fixes `HfArgumentParser.__init__` when `dataclass_types` is a generator. Without this, when a generator is passed, after the init it's exhausted so then a call to `parse_args_into_dataclasses` will return a wrong result.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

I think pretty much anyone.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[M2M100, XGLM] fix create_position_ids_from_inputs_embeds
Author: patil-suraj
Pull request: yes

# What does this PR do?

The `create_position_ids_from_inputs_embeds` method does not use the `past_key_values_length` when creating `input_ids` , which creates issues in generation when `use_cache=True`. This PR fixes the `create_position_ids_from_inputs_embeds` method.

Fixes #15736

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Package transformers.onnx should handle Tensorflow
Author: Albertobegue
Pull request: no
Labels: name: WIP

# 🚀 Feature request

The package transformers.onnx should allow the conversion of tensorflow models as well as Pytorch's. 

## Motivation

[This documentation](https://huggingface.co/transformers/serialization.html) features the following code snippet 

```

python -m transformers.onnx --help

usage: Hugging Face ONNX Exporter tool [-h] -m MODEL -f {pytorch} [--features {default}] [--opset OPSET] [--atol ATOL] output

positional arguments:
  output                Path indicating where to store generated ONNX model.

optional arguments:
  -h, --help            show this help message and exit
  -m MODEL, --model MODEL
                        Model's name of path on disk to load.
  --features {default}  Export the model with some additional features.
  --opset OPSET         ONNX opset version to export the model with (default 12).
  --atol ATOL           Absolute difference tolerance when validating the model.
```

which introduces the new package transformers.onnx. It also says: "This conversion is handled with the PyTorch version of models - it, therefore, requires PyTorch to be installed. If you would like to be able to convert from TensorFlow, please let us know by opening an issue." so here I am, opening an issue :) 


I have been converting Tensorflow models with ONNX for a while but I am not observing the same time-saving as expected and described by various references, mainly using Pytorch though (like this one: https://medium.com/microsoftazure/accelerate-your-nlp-pipelines-using-hugging-face-transformers-and-onnx-runtime-2443578f4333). Therefore, I am eager to try this new package for my conversions but I cannot currently because we use Tensorflow.

## Your contribution

I would be very keen to contribute to this development but as it would be my first contribution to transformers I would very much appreciate any guidance on how to do it and what to read!

Thank you :))


Comments:
Thank you for the feature request! We'll prevent the stale bot from closing that PR and see if there is a lot of interest in this addition.

Closing this since PR #13831 implemented the feature :)
[Bug] Gradient Checkpointing In CLIP Model
Author: fuliucansheng
Pull request: no

https://github.com/huggingface/transformers/blob/master/src/transformers/models/clip/modeling_clip.py#L575 

Hi, there, 
&nbsp;&nbsp;&nbsp;&nbsp;I believe `hidden_states = layer_outputs[0]` should be in the for loop. I try to enable the gradient checkpointing for the clip model but find this bug.

Comments:
Great catch! Fix is here https://github.com/huggingface/transformers/pull/15789
Add Model Interpretability Module
Author: Alymostafa
Pull request: no

# 🚀 Feature request
What about adding an Interpretability Module like integrating the captum with hugging face, because there are a lot of difficulties when trying to use captum with hugging face transformer models





Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.