noamaanMulla-03/transformers-issues huggingface dataset:
Inconsistent PreTrainedTokenizerBase.pad  argument default value & docstring
Author: Fraser-Greenlee
Pull request: no

The docstring states the argument `padding` has a default of `False` but its default is `True`

docstring: 
https://github.com/huggingface/transformers/blob/d5b3e56de5376aa85ef46e7f0325139d9e299a41/src/transformers/tokenization_utils_base.py#L2469-L2470
arg:
https://github.com/huggingface/transformers/blob/d5b3e56de5376aa85ef46e7f0325139d9e299a41/src/transformers/tokenization_utils_base.py#L2431-L2472

This causes issues when using `DataCollatorForLanguageModeling` with an already padded dataset as it resets the attention mask.

Comments:
Seems to have been added in this commit: 
https://github.com/huggingface/transformers/commit/f3065abdb8805f5beaed9ff1e92ce874e655f5c9#diff-85b29486a884f445b1014a26fecfb189141f2e6b09f4ae701ee758a754fddcc1R2146-R2168
As part of merge https://github.com/huggingface/transformers/pull/6110

Hi, indeed! The docs should be changed to reflect the method signature. Do you want to open a PR?
[Examples] fix few typos in help messages and arguments
Author: baeseongsu
Pull request: yes

# What does this PR do?

- fix typos in help message
- consistently use `gpus`, instead of `n_gpu` (followed by #6315)
(it's not working because not fully converted)


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

@VictorSanh

Comments:
Hi @baeseongsu 
thanks for opening this. I am actually working on a fairly big PR to revamp these distillation scripts. I'll integrate your modifications directly there to have everything in a single place!
I **hope** to do this by end of week
Victor
[finetune_trainer] --evaluate_during_training is no more
Author: stas00
Pull request: no

In `examples/seq2seq/builtin_trainer/` all scripts reference `--evaluate_during_training ` but it doesn't exist in pt trainer, but does exist in tf trainer:
```
grep -Ir evaluate_during
builtin_trainer/finetune.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
builtin_trainer/train_distil_marian_enro.sh:    --do_train --do_eval --do_predict --evaluate_during_training\
builtin_trainer/finetune_tpu.sh:    --do_train --do_eval --evaluate_during_training \
builtin_trainer/train_distilbart_cnn.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
builtin_trainer/train_distil_marian_enro_tpu.sh:    --do_train --do_eval --evaluate_during_training \
builtin_trainer/train_mbart_cc25_enro.sh:    --do_train --do_eval --do_predict --evaluate_during_training \
```

```
Traceback (most recent call last):
  File "finetune_trainer.py", line 310, in <module>
    main()
  File "finetune_trainer.py", line 118, in main
    model_args, data_args, training_args = parser.parse_args_into_dataclasses()
  File "/home/stas/anaconda3/envs/main-38/lib/python3.8/site-packages/transformers/hf_argparser.py", line 144, in parse_args_into_dataclasses
    raise ValueError(f"Some specified arguments are not used by the HfArgumentParser: {remaining_args}")
ValueError: Some specified arguments are not used by the HfArgumentParser: ['--evaluate_during_training']
```
Is this meant to be replaced by: `--evaluation_strategy` - this is the closest I found in `training_args.py`

If so which one? `steps` or `epoch`?

Also the help output is borked:

```
$ python finetune_trainer.py -h
...
 [--evaluation_strategy {EvaluationStrategy.NO,EvaluationStrategy.STEPS,EvaluationStrategy.EPOCH}]
```
probably this is not what what's intended, but 
```
 [--evaluation_strategy {no, steps, epochs}
```

But perhaps it's a bigger issue - I see `trainer.args.evaluate_during_training`:
```
src/transformers/integrations.py:        ) and (not trainer.args.do_eval or not trainer.args.evaluate_during_training):
```
and also `--evaluate_during_training` in many other files under `examples/`.

Thank you.

@sgugger, @patrickvonplaten 


Comments:
Found the source of breakage: https://github.com/huggingface/transformers/pull/8604 - I guess that PR needs more work
Remove deprecated `evalutate_during_training`
Author: sgugger
Pull request: yes

# What does this PR do?

Replaces the `evaluate_during_training` in examples using the `Trainer` (as well as integrations and tf_trainer) by the new `evaluation_strategy`.

Fixes #8792

Comments:
Merging this because we need it for the v4.0.0, pinging @jplu so he's aware of the changes made to the TFTrainer.
token-classification: use is_world_process_zero instead of  is_world_master()
Author: stefan-it
Pull request: yes

Hi,

I just found some leftovers of the `is_world_master()` function in the token classification example.

As this method has been removed, the following error message is thrown when using the `do_prediction` option:

```bash
Traceback (most recent call last):
  File "run_ner.py", line 394, in <module>
    main()
  File "run_ner.py", line 372, in main
    if trainer.is_world_master():
AttributeError: 'Trainer' object has no attribute 'is_world_master'
```

This PR fixes it and uses the new `is_world_process_zero()` method instead!

Comments:
/cc @sgugger :hugs:
Attempt to fix Flax CI error(s)
Author: mfuntowicz
Pull request: yes

- Increased the tolerance when comparing Flax et PyTorch output (_~0.00058 on my dev box_)
- Removed the `jit` parametrization when running `test_multiple_sentences` because it leads to instabilities
- Introduced subtests expliciting what we're doing by enabling / disabling JIT. 

Comments:
Better filtering of the model outputs in Trainer
Author: sgugger
Pull request: yes

# What does this PR do?

As discovered since merging #8530, sometimes (e.g. when using nvidia apex with the O2 optimization) the new model outputs lose their type and become regular dictionaries. This means we can't index into them with integers and some rework in the internals of `Trainer` has become necessary.

This PR:
- fixes the training by indexing in the outputs by string if they are dict, int otherwise when grabbing the loss
- fixes the evaluation by indexing in the outputs by string if they are dict, int otherwise when grabbing the loss

but it also takes advantage of the new dict outputs to better filter the outputs at inference. We had several issues recently when using models outputing past states (such as Reformer, XLNet, GPT-2) during evaluation in `Trainer`. This PR introduces a new API that looks at a possible key in the config of the model to get some attributes to ignore in the ouputs during evaluation (those outputs are then discarded from the predictions returned by the function `Trainer.predict` or passed along to metric computation in `Trainer.evaluate`). Since a user might have some use cases where they want to ignore more keys or output those keys, a new argument is added to both `Trainer.predict` and `Trainer.evaluate` to fully control the keys ignored in those dictionaries.

If the model outputs tuple, this is all ignored.

Fixes #8523 among others


Comments:
Ctrl for sequence classification
Author: elk-cloner
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes #7623


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 @LysandreJik 
 -->


Comments:
Same as GPT-2, this would benefit from also handling padding on the left; I'll work on this in another PR.
Transfoxl sequence classification
Author: spatil6
Pull request: yes

This PR implements Sequence classification for Transformer XL model
TransfoxlForSequenceClassification uses the last token in order to do the classification, as other causal models (e.g. GPT-1,GPT-2) do.

Fixes #7623 (Partially)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?
@LysandreJik




Comments:
Same as GPT-2, this would benefit from also handling padding on the left; I'll work on this in another PR.

@LysandreJik , I'll raise new PR, there was some conflicts in it.
can the BertModel convert to onnx? whether any one had done sucessfully ?
Author: ghost
Pull request: no


Comments:
2 resources in the thread linked by valhalla: https://discuss.huggingface.co/t/how-to-apply-pruning-on-a-bert-model/1658/5

> 2 resources in the thread linked by valhalla: https://discuss.huggingface.co/t/how-to-apply-pruning-on-a-bert-model/1658/5

Thank you very much for your warm reply, I will study it.
What would be the license of the model files available in Hugging face repository?
Author: jaingaurav3
Pull request: no

Dear Team,

Could you clarify what would be the license for different models pushed to hugging face repo like legal_bert, contracts_bert etc? Would the model file follows the same license i.e. Apache 2.0 like the hugging face library?

Regards
Gaurav

Comments:
Maybe @julien-c can answer!

You can check on the model hub:
eg. apache-2.0 models: https://huggingface.co/models?filter=license:apache-2.0
mit models: https://huggingface.co/models?filter=license:mit

etc.

Thanks @julien-c for the update.
2 typos in modeling_rag.py
Author: ratthachat
Pull request: yes

# What does this PR do?
Fix 2 typos in `modeling_rag.py`
`from_encoder_generator_configs` --> `from_question_encoder_generator_configs`

## Who can review?
@lhoestq


Comments:
Hi! Could you run `make style` on your branch so that the code quality check passes? Thanks!

Hi guys, I only have mobile phone until Dec. 1. I will do it as soon as I can access PC.

@lhoestq @LysandreJik done applying style. Sorry for late!
Make the big table creation/check platform independent
Author: sgugger
Pull request: yes

# What does this PR do?

As @jplu mentioned in #8813, the check that the big table of models/tokenziers is up-to-date (done in `make quality`) requires all three backends installed (plus tokenizers and sentencepiece). This PR amends the script to use the objects in the init (that are always there, thanks to the dummies) instead of the dicts in the auto modules (which are set to None if a specific backend is not installed).

In passing, it adds aliases `MT5Tokenizer` and `MT5TokenizerFast` (to `T5Tokenizer` and `T5TokenizerFast` respectively) because otherwise the script does not detect the tokenizers associated to this model, cc @patrickvonplaten 

Comments:
length_penalty not influencing results (Bart, Pegasus)
Author: marcoabrate
Pull request: no

Hello,

I am experimenting with the generative parameters of the two models Bart and Pegasus. In particular, I am having trouble with the `length_penalty` parameter, since changing it does not change the output of the model.

I am summarizing two different chapters of a book (# tokens around 1k) and this is the code I am using:

```
    model.generate(
        b0ch1sec1_text_enc,
        min_length = 150,
        max_length = 350,
        num_beams = 2,
        length_penalty = lp,
        early_stopping = True)[0]
```

With `lp` swiping from 0.1 to 2 and model being either `bart-large-cnn` or `pegasus-large`.

Do you have any idea why the output does not change at all?

Comments:
Hello, thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the [forum](https://discusss.huggingface.co) instead?

Thanks!
Better support for resuming training
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds two things linked to resuming training:

1. It brings full reproducibility when resuming an interrupted training from a checkpoint (i.e., resuming a training from a checkpoint will give the exact same results as a training from the beginning with the same seeding). This was not currently the case because the dataloader shuffle was not triggered `epochs_already_trained` times, so the shuffle of the dataloader was the same as epoch 0. So the full reproducibility was only there for trainings resumed from an early checkpoint (during the first epoch).

2. It also adds the option to ignore that data skipping which can take a very long time on a large dataset. This will go faster but yield different results from a training from scratch.

Fixes #8874 and #8876

Comments:
[trainer] add distributed_env to TrainingArguments
Author: stas00
Pull request: no

As discussed in https://github.com/huggingface/transformers/pull/8823 it's not simple to check whether the downstream code is running under distributed mode or not (currently requires checking `self.args.local_rank != -1` which is far from obvious.

So we were discussing about adding a flag like `distributed_env` so that the downstream code could do a much simpler intuitive check.

I'm not sure whether we need just True/False for ddp or whether we also need to have another flag if we are under DP as well?

@sgugger 

Comments:
Add a `parallel_mode` property to TrainingArguments
Author: sgugger
Pull request: yes

# What does this PR do?

This PR adds a `distributed_env` property to the `TrainingArugments` making it clear if we are in:
- a single process (CPU or one GPU)
- a parallel setting (one process but several GPUs)
- a distributed parallel setting (several processes, one per GPU)
- a TPU setting

Fixes #8858


Comments:
Given our discussion yesterday, I'm not sure `distributed_env` is fitting. As you convinced me that DP is not distributed when it comes to pytorch conventions, `if self.distributed_env == "dp"` is back to being confusing.

Given that with the exception of tpu, all dp/ddp/mp/pp are SomethingParallel, should it be called `parallel_mode`?

I don't know anything about tpu, so it's hard for me to know where it fits. But it's probably not distributed either. And not parallel either.

So perhaps we call it `compute_env`

LGTM, @sgugger!
Prevent BatchEncoding from blindly passing casts down to the tensors it contains
Author: Craigacp
Pull request: yes

# What does this PR do?

This PR prevents `BatchEncoding.to` from passing down things which aren't devices to the tensors it contains. Previously it would pass down all the arguments, and as the `to` method in pytorch can also cast the arguments to different types it's used blindly by other packages (e.g. Nvidia's Apex). This caused an issue where when using Apex's AMP support with `O2` or greater it would cast the token indexes from a `LongTensor` to a `HalfTensor` truncating our vocab at 65k and rounding most of the words to the nearest 8th word (if you blindly insert the cast back in in the embedding layer, which the warning says to do).

The doc for `BatchEncoding.to` says it is only for moving the encoding and the tensors it contains between devices, but as the type checking isn't on by default it can behave like a regular pytorch `to` method and accept cast arguments that it passes down to the tensors it contains.

Fixes #6582

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case. #6582
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

There are no docs or tests changes as the change makes the method conform with its currently documented behaviour.

@LysandreJik



Comments:
black complained about the style after the update, so I fixed it and squashed the commits again.

Thank you @Craigacp!
[trainer] start using training_args.parallel_mode
Author: stas00
Pull request: yes

Following up on https://github.com/huggingface/transformers/pull/8877 which adds `training_args.parallel_mode` to make it easy to comprehend which mode the trainer is running under - this PR deploys the new property in a few places.

@sgugger, have I deployed it as you envisioned it?



Comments:
Thank you for adding this new property, @sgugger - it has indeed improved the readability!
How to pass the attention mask as a param to model forward when using torchscript?
Author: JiayiFu
Pull request: no

## Environment info

- `transformers` version:
- Platform: Ubuntu 16.04
- Python version: 3.6.9
- PyTorch version (GPU): 1.3.0+cu100
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

## Information
 I am using a Bert model downloaded from hugging face. I finetuned that model for a two-class classification task and convert that model to a `torchscript` through the `jit.trace`.



The following code show how I got the torchscript:

    from transformers import BertTokenizer, BertForSequenceClassification
    tokenizer = BertTokenizer.from_pretrained(tokenizer_dir)
    model = BertForSequenceClassification.from_pretrained(model_dir, num_labels=2, torchscript=True)
    model.eval()
    model = model.to("cuda:0")
    
    input_text = ["test this case", "test test this case"] 
    encoding = tokenizer(text_batch, return_tensors='pt', padding=True, truncation=False)
    input_ids = encoding['input_ids']
    attention_mask = encoding['attention_mask']
    
    tokens_tensor = torch.tensor(input_ids)
    tokens_tensor = tokens_tensor.to("cuda:0")
    traced_model = torch.jit.trace(model, tokens_tensor)
    torch.jit.save(traced_model, str(pt_path))


The following code show how I use the torchscript, the input tensor is the same with the first code:

    pt_model = torch.jit.load(model_path))
    pt_model.eval()
    pt_label = pt_model(input_tensor)[0]


For the normal model, it needs to pass two params input tensor and attention mask like:

    model(input, attention_mask=attn_mask)


But for the `torchscript`, I can't pass the attention mask to the model.
So, what's the right way to use `torchscript` to do the forward with the attention mask?

Thanks!




Comments:
I think you would need to compile your model with both the tokens tensor and the attention mask. Given that the attention mask is the second argument, you can pass it directly when tracing the model:

```py
traced_model = torch.jit.trace(model, [tokens_tensor, attention_mask])
```

then you can do:
```py
model(tokens_tensor, attention_mask)
```

@LysandreJik It works. Thanks for your help!
custom prepare_inputs_for_generation for generation 
Author: rabeehkarimimahabadi
Pull request: no

Hi
I need to change  model_inputs used for the generation, I am using T5ForConditionalGeneration which has extra input parameter and this needs to be passed in each time I call model.generate(), I cannot see how to rewrite the generate function to also pass this argument, could you provide me with some explanation: 

https://github.com/huggingface/transformers/blob/d5b3e56de5376aa85ef46e7f0325139d9e299a41/src/transformers/generation_utils.py#L676

thanks


Comments:
solved with implementing it inside the model of T5ForConditionalGeneration  thanks