noamaanMulla-03/transformers-issues huggingface dataset:
Trainer accepts iterable datasets
Author: j-rossi-nl
Pull request: yes

Fixes #5990 
Follows PR #5995 (got closed for being stale...)

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [X] Did you write any new necessary tests? 


## Who can review?
@sgugger @LysandreJik 



Comments:
Hi there, thanks for the PR! Unfortunately it's not based on current master, so we can't review properly as it suggests changes from code that does not exist anymore.

Some merging work to be done.
consolidate tf activation functions
Author: sshleifer
Pull request: no
Labels: name: Help wanted

name: cleanup

e.g.
```
def gelu(x):
    """Gaussian Error Linear Unit.
    This is a smoother version of the RELU.
    Original paper: https://arxiv.org/abs/1606.08415
    Args:
        x: float Tensor to perform activation.
    Returns:
        `x` with the GELU activation applied.
    """
    cdf = 0.5 * (1.0 + tf.tanh((np.sqrt(2 / np.pi) * (x + 0.044715 * tf.pow(x, 3)))))
    return x * cdf
```
See ACT2FN for torch.


Comments:
Done by @jplu ! https://github.com/huggingface/transformers/blob/master/src/transformers/activations_tf.py#L50
[testing] fix/hide warnings
Author: stas00
Pull request: yes

in `test_trainer_callback.py`:

- fixes PytestCollectionWarning - pytest doesn't like non-unittest classes starting with `Test`
- hides scatter_gather warnings - as it was suggested they are just so

@sgugger 

Fixes: https://github.com/huggingface/transformers/issues/7832


Comments:
trainer/lightning_base: Arbitrary config updates through command line
Author: sshleifer
Pull request: no
Labels: name: wontfix

this issue [https://github.com/huggingface/transformers/issues/6367], and a recent one to add dropout to the command line, as well as the usage of task_specific_params during finetuning, are all one off solutions to address a larger problem. During finetuning/training, it is very difficult to arbitrarily set config attributes. For `examples/lightning_base.py`, you need to save a whole new config to json and put it in a directory, which is fairly annoying method for changing hyperparameters, so we add lots of them, like `--dropout --attention_dropout --encoder_layerdrop --decoder_layerdrop` through `argparse.add_argument`. 
It would be a better user experience if I could just pass any kwarg without editing the code.

This seems possible with the `fire` package. But I would prefer an `argparse` solution as there is another issue open to delete the `fire` dependency, I also asked a similar question on
[stackoverflow](https://stackoverflow.com/questions/63329044/python-argparse-allow-unregistered-arguments)

Comments:
see `argparse.remainder` https://stackoverflow.com/questions/22850332/getting-the-remaining-arguments-in-argparse/46250042#46250042

h/t @stas00 



You could probably extract all the known args via `argparse`'s:
```
args, unknown = parser.parse_known_args()
```
and then use another tool to parse `unknown` (which is just `argv` minus known args) e.g. could even use `fire` or write a core function to do that.

surely a cheat, but it would make your query work, while having `arparse` as the main solution still.

You could also take a look at https://github.com/huggingface/transformers/blob/155288f04ba9a5d0a0e4d5be4f6d4e808ad8cfff/src/transformers/hf_argparser.py#L128-L146

If I read the code correctly, it doesn't do anything with `remaining_args`. It either just returns them as is (`argv` list) or throws an error.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Fixed by @stas00 for both trainers, thanks!
Generate coherent text with T5
Author: parthplc
Pull request: no

Can we use T5 to generate coherent text given an input line like in case of GPT-2? If yes how do we fine-tune T5 for such a task?


Comments:
@parthplc Did you find an answer to this question?

@jsrozner yeah, I was able to finetune t5 for text generation.

> @jsrozner yeah, I was able to finetune t5 for text generation.

Is your work publicly visible anywhere? (colab or a repository) I was looking for some examples to adapt to my use case. Although I got a helpful response here: https://discuss.huggingface.co/t/t5-for-conditional-generation-getting-started/1284/2

I used the same @jsrozner 

@parthplc @jsrozner can someone share a nb of using finetune.py + t5, would love to see some examples for reference. 

Check out 
https://discuss.huggingface.co/t/t5-seq2seq-custom-fine-tuning/1497/6 (details on tweaking your vocab if you need it)

The link above (https://discuss.huggingface.co/t/t5-for-conditional-generation-getting-started/1284) has links to other useful T5 posts on huggingface.

And I am looking for answers to this post for some other tips and tricks: 
https://discuss.huggingface.co/t/t5-tips-for-finetuning-on-crossword-clues-clue-answer/1514

To actually run, you should do this (called "installing from source):
- clone transformers repository
- create new conda or other env
- (from top level directory), pip install -e .
- cd examples && pip install -r requirements.txt
- cd seq2seq
- ./finetune_t5_bart_tiny.sh (or whatever the name is) -- will run two test epochs

After that it's just a matter of choosing the model, tokenizer, setting up your data, and then tweaking params. Most of that is in the posts I linked!
Update pyarrow to meet datasets 1.1.2
Author: ncoop57
Pull request: yes

# What does this PR do?

Update pyarrow to meet the requirement of y'alls awesome datasets 1.1.2 library for running the examples (tested on the finetune_tiny_bart.sh seq2seq example). You can see the colab I tested on here: https://colab.research.google.com/drive/12XSQoFRlpXLEd_tfvwC5wUVYJZUPKFwE?usp=sharing


Fixes #7691


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Issue #7691
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.
@sshleifer 

Comments:
The `datasets` library already has a requirement on `pyarrow>=0.17.1`: https://github.com/huggingface/datasets/blob/master/setup.py#L68, so this line should not be necessary!

OH whoops, this must be a colab thing since it already has a `pyarrow` version installed, so what I needed to do was run the pip install with the [`--ignore-installed` flag](https://stackoverflow.com/questions/24764549/upgrade-python-packages-from-requirements-txt-using-pip-command).
Training Data of  xlm-roberta-large-finetuned-conll03-* models
Author: wangxinyu0922
Pull request: no

Hi, I'm curious about the training data of xlm-r models finetuned on conll ner datasets (e.g.  xlm-roberta-large-finetuned-conll03-german,  xlm-roberta-large-finetuned-conll03-english), are the models trained on train+dev sets?

Comments:
Pinging @stefan-it 

Hi @wangxinyu0922 ,

the models are only trained on the corresponding training data sets, that means development data was not used for training :)

That's great! Thank you!

> 
> 
> Hi @wangxinyu0922 ,
> 
> the models are only trained on the corresponding training data sets, that means development data was not used for training :)

@stefan-it 
By the way, what is the accuracy of the model on the four datasets? The models are trained on document context or sentence context? I believe different context will affect the performance.
modify ``val_loss_mean``
Author: xujiaze13
Pull request: yes
Labels: name: wontfix

revised by this warning
***/lib/python3.*/site-packages/pytorch_lightning/utilities/distributed.py:25: RuntimeWarning: The metric you returned 1.234 must be a `torch.Tensor` instance, checkpoint not saved HINT: what is the value of val_loss in validation_epoch_end()?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[WIP] Add Pegasus
Author: JingqingZ
Pull request: yes
Labels: name: wontfix

name: Summarization

* Add PEGASUS in TF2

Comments:
Hi @sshleifer, I will continue to push code to this PR and make it runnable asap.

@sshleifer The code has been uploaded. (1) The test is runnable in TF2 and loads AESLC checkpoints successfully with correct outputs. (2) Code of models and layers (including decoding, beam search) are all in a single file which may look messy (sorry). (3) Most code is simply copied and pasted (then converted to TF2) from the original PEGASUS repo so you may refer to the original repo for clearer code if necessary. 

I think you can start from here. Please let me know if I can help further.

@JingqingZ I'm gunna add torch first in #6340 (you will be a PR co-author). And then come back here to finish TF. No action needed from you, just an update.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Onnx + TensorRT uses CPU not GPU
Author: agemagician
Pull request: no

## Environment info

- `transformers` version: 3.1.0
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): 2.3.0 (True)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help
@mfuntowicz 

## Information

Model I am using Bert:

The problem arises when using:
* [X] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

https://colab.research.google.com/drive/1mqn99U2NMm-fdw213h0xsqm_uRwRkNzp?usp=sharing

## Expected behavior

I am trying to run onnx using tensorrt on the gpu, but it always uses the CPU.
I tried to install different version or onnx and cuda with not luck.
Any idea why "TensorrtExecutionProvider" runs on the CPU rather than GPU ? and how to fix it ?


Comments:
@mfuntowicz - sorry to ping you here :D Do you have a "high level" idea why this might not work?

@agemagician IIUC TensorRT needs a GPU with Tensor cores to work correctly. You're using a P100 in the notebook which doesn't have those. So Onnx will fall back to using CUDAExecutionProvider. Now that itself may be slow because of [this](https://github.com/microsoft/onnxruntime/blob/master/docs/ONNX_Runtime_Perf_Tuning.md#why-is-my-model-running-slower-on-gpu-than-cpu). I'd suggest setting verbose logging level and checking. To enable set the following:
```
options = SessionOptions()
options.log_verbosity_level=2
options.log_severity_level=0
```
Then in the console logs if you see something like
`2020-10-16 20:25:31.386914373 [W:onnxruntime:Default, fallback_cpu_capability.h:140 GetCpuPreferedNodes] Force fallback to CPU execution for node: Gather_9`, or there's a huge list
```
2020-10-16 20:25:31.406073394 [V:onnxruntime:, inference_session.cc:869 TransformGraph]  Provider: [CPUExecutionProvider]: [Gather (Gather_9), Unsqueeze (Unsqueeze_10), DynamicQuantizeLinear (237_QuantizeLinear), QAttention (Attention_qua...
```
 that indicates that some parts of your model are not GPU compatible because Onnx doesn't have the supported operators.

In my case it was happening because I was using a quantized model with the CUDA provider, but the quantized model needs TensorRT to run I believe.

Thanks @bdalal  for your reply. I have tested it also on V100 but I had the same issue.
Anyway, I think the benefits vs troubles, seems to be not worth it for now.
I hope Nvidia, onnx and Pytorch provide a better integration on the future for tensor rt.
Fix Rag example docstring
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7829


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), 
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to the it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests? 


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik 
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
Remove duplicated mish activation function
Author: Razcle
Pull request: yes

# What does this PR do?
The mish activation function was repeated in the file. I removed the duplicate.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dimiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik 
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 Translation: @sshleifer
 Summarization: @sshleifer
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @sshleifer
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @sshleifer
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 -->


Comments:
i have used t5_base for abstractive summarization but it is not giving good results,Could you please give me solution for this
Author: gopal354
Pull request: no
Labels: name: wontfix

# 🖥 Benchmarking `transformers`

## Benchmark

Which part of `transformers` did you benchmark?

## Set-up

What did you run your benchmarks on? Please include details, such as: CPU, GPU? If using multiple GPUs, which parallelization did you use?

## Results

Put your results here!


Comments:
Hi @gopal354 , this depends on lot of factors. What is the domain of your dataset ? There are many other summrization models  available on the model hub trained on different datasets. You can try them as well. Or if you have a dataset, then you can further fine-tune these models on your domain. 

Would be nice if the detailed question is written in the description box rather than title and use the relevant issue topic (this should be Questions & Help and not Benchmarking transformers). This will help the team and contributors to act faster on the issue :) 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Adding example with Finnish BERT fine-tuning for NER task
Author: bmichele
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Add an example in `/examples/token-classification`. The example consists in the fine-tuning of the Finnish BERT models (`bert-base-finnish-cased-v1` and `bert-base-finnish-uncased-v1`) with the FiNER dataset (https://github.com/mpsilfve/finer-data).

The example is similar to the already existing GermEval and WNUT'17 examples, but for Finnish language.

## Motivation

The fine-tuned model can be used to extract product, location, date, event, organization and person entities from Finnish sentences.
The scripts necessary for the fine-tuning are already available [here](https://github.com/bmichele/transformers/tree/finnish-ner). I would like to contribute to the transformer projects and get feedback from the community on this. Note that I submitted a PR but this was never reviewed and finally closed by stale bot (see https://github.com/huggingface/transformers/pull/3474).

## Your contribution

I can adapt the code in the closed PR in order to be merged in master.


Comments:
Hi @bmichele , 
The `/examples/token-classification` is pretty generic, you won't need to add new example. If you process the data in the format expected by the example the you can pass the `bert-base-finnish-cased-v1` model using the model_name_or_path argument.

Hope this helps.

Hi @patil-suraj , thank you for your input.

As I mentioned in the issue, the example for Finnish is already implemented in this branch:
https://github.com/bmichele/transformers/tree/finnish-ner

The branch contains the scripts necessary to
 * download and preprocess the FiNER dataset
 * perform the fine-tuning

Moreover, I added the results in the readme file:
https://github.com/bmichele/transformers/blob/finnish-ner/examples/ner/README.md

I see that few months ago the WNUT’17 example was added, my question is if there is interest in merging also the Finnish example. If this is the case, I will take care of updating it so that it can be merged without conflicts and create a PR.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Using tensorrt model.engine Inference speed is relatively fast. Why is onnxruntime based on tensorrt as slow as CPU inference
Author: ye1024
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help
https://github.com/huggingface/transformers/blob/master/notebooks/04-onnx-export.ipynb
Based on different providers to infer the  .onnx model, why is CUDA the fastest and tensorrt based model particularly slow
Using tensorrt model.engine Inference speed is relatively fast. Why is onnxruntime based on tensorrt as slow as CPU inference


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
inconclusive truncation strategies in encode_plus?
Author: fhamborg
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

## Details

Not sure if this is a bug or missing feature or I just misunderstood something: specifically, I'm wondering whether a common truncation strategy is missing from [`encode_plus`](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.__call__). More specifically, if you have a input sequence pair consisting of two texts `a` and `b` and invoke `encode_plus` as follows:

``` 
encode_plus(text=a, text_pair=b, max_length=100, truncation=whatever)
```

It seems that there is no truncation strategy that simply cuts of tokens from the end of the (internally concatenated) input sequence consisting of a and b. Instead the three options allow either to truncate from only a, from only b, or from the longest first (which could be either a or b, depending on the input). How can one remove token by token from the right until max_length is reached, e.g., also in the case that len(a)=200 and len(b)=2. In this example, no option seems to be suitable, e.g., "longest first" would remove only from a, "only first" likewise only from a (both of these remove from the first input a, but should in my case remove from end of the total sequence, e.g., first b then a if necessary), and "only second" only from b (which would be removed entirely, but since only second is defined, a will not be truncated so the total length is 200 and thus still longner than max length)

SO link: https://stackoverflow.com/questions/63280435/huggingface-transformers-truncation-strategy-in-encode-plus

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.