noamaanMulla-03/transformers-issues huggingface dataset:
Add option to resize like torchvision's Resize
Author: NielsRogge
Pull request: yes

# What does this PR do?

A lot of research papers (like ConvNeXT, PoolFormer, etc.) use [`torchvision.transforms.Resize`](https://pytorch.org/vision/stable/transforms.html#torchvision.transforms.Resize) when preparing images for the model. This class supports providing an integer size, which will only resize the smaller edge of the image.

The `resize` method defined in `image_utils.py` currently resizes to (size, size) in case size is an integer. This PR adds the possibility to resize images similar to torchvision's resize by adding an argument `torch_resize` (which is set to `False` by default). One can then also provide an optional `max_size` argument.

This is replicated using Pillow.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Applied all suggestions from your review, I've renamed `torch_resize` (which defaults to `False`) to `default_to_square` (which defaults to `True`).

I guess we cannot default it to a rectangle, to be backwards compatible? Or can we, if we change the existing ones that expect a square to `default_to_square=True`)? Based on [this](https://github.com/huggingface/transformers/search?q=self.resize), it seems like we currently have 10 models using the resize method that assume providing an integer will resize to a square.

>  I guess we cannot default it to a rectangle, to be backwards compatible? Or can we, if we change the existing ones that expect a square to default_to_square=True)? Based on this, it seems like we currently have 10 models using the resize method that assume providing an integer will resize to a square.

We can switch the other way, but only if we change the default for all the subclasses that expected the resize to a square, so not sure it's worth the hassle.
add custom stopping criteria to human eval script
Author: lvwerra
Pull request: yes

This PR adds the new custom `stopping_criteria` feature in `generate` (#14779) to the code evaluation script at `scripts/human_eval.py`.

In HumanEval the task is to complete a function and with this feature the generation is stopped once the function body is complete. To do this a set of keywords, named end-of-function (EOF), is used. If any of these EOF keywords appear in the generation the function body must be finished. E.g. if `\ndef` is in the generated sequence this means that the current body was completed.

This feature increases the generation speed for HumanEval by 2-3x (from 8h to ~3h).

Comments:
Human eval crashes for me, and I believe this PR is to blame. Here's the stack:

```
Traceback (most recent call last):
  File "human_eval.py", line 126, in <module>
    main()
  File "human_eval.py", line 106, in main
    task_generations.extend(complete_code(pipe, prompt, num_completions=args.batch_size, **gen_kwargs))
  File "human_eval.py", line 50, in complete_code
    code_gens = pipe(prompt, num_return_sequences=num_completions, **gen_kwargs)
  File "/opt/conda/lib/python3.7/site-packages/transformers/pipelines/text_generation.py", line 150, in __call__
    return super().__call__(text_inputs, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/transformers/pipelines/base.py", line 924, in __call__
    return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
  File "/opt/conda/lib/python3.7/site-packages/transformers/pipelines/base.py", line 931, in run_single
    model_outputs = self.forward(model_inputs, **forward_params)
  File "/opt/conda/lib/python3.7/site-packages/transformers/pipelines/base.py", line 880, in forward
    model_outputs = self._forward(model_inputs, **forward_params)
  File "/opt/conda/lib/python3.7/site-packages/transformers/pipelines/text_generation.py", line 165, in _forward
    generated_sequence = self.model.generate(input_ids=input_ids, **generate_kwargs)  # BS x SL
  File "/opt/conda/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/transformers/generation_utils.py", line 1027, in generate
    **model_kwargs,
TypeError: sample() got multiple values for keyword argument 'stopping_criteria'
```

I'm using Transformers 4.12.2. It seems like `sample` call in `generation_utils.py` already has `stopping_criteria` keyword argument set. 

Indeed, this requires `transformers==4.15.0`, where handling of custom stopping criteria was introduced. See changes to `requirements.txt`. Hope upgrading works!
cannot import name 'RobertaForTokenClassification'
Author: dsindex
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): BERT

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [x] the official example scripts: (give details) examples/run_ner.py
* [ ] my own modified scripts: (give details)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: (give the name) NER on CoNLL2003 ENG
* [ ] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

1. $ pip install transformers
2. $ python
3. > from transformers import RobertaConfig, RobertaForTokenClassification, RobertaTokenizer
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'RobertaForTokenClassification'

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Ubuntu 18.04.3
* Python version: 3.6
* PyTorch version: 1.2.0
* PyTorch Transformers version (or branch): pip install transformers
* Using GPU ? Yes, CUDA 10
* Distributed of parallel setup ? No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
+ other bug
  - with '--evaluate_during_training'
```
File "/path-to/run_ner.py", line 167, in train
    results, _ = evaluate(args, model, tokenizer, labels, pad_token_label_id)
TypeError: evaluate() missing 1 required positional argument: 'mode'
```

- update
  - using `pip3 install git+https://github.com/huggingface/transformers.git --upgrade` command, the first bug got away. 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


I am also facing the same issue, but even installing from git (as stated by dsindex) did not help.

It could be because torch is not installed. transformers doesn't install torch automatically but needs the same to load the models. try `pip install torch` and import again!

I am having the same issue, neither installing from git, nor `pip install torch` have fixed the issue

Could you provide your software versions, by running `transformers-cli env` in your environment?
Save code of registered custom models
Author: sgugger
Pull request: yes

# What does this PR do?

This PR changes the way `save_pretrained` works for user custom objects when they are registered to an auto API via the the new command `register_for_auto_class`. Here is an example:

```py
# Custom config
from test_model.configuration import NewModelConfig
# Custom model
from test_model.modeling import NewModel

# No need to specify AutoConfig here, that's the default and the only auto class that makes sense for a config.
NewModelConfig.register_for_auto_class()
# AutoModel is also the default but we show it here as you could use an other auto model class.
NewModel.register_for_auto_class("AutoModel")

# Just an example of model definition
model = NewModel.from_pretrained("hf-internal-testing/tiny-bert")
model.save_pretrained("test-dynamic")
# Model weights, config and the files with the code of NewModel and NewModelConfig are copied in this folder
```

If the user pushes the folder to the Hub, anyone will then be able to use the Auto api to load this model and config (as long as they pass `trust_remote_code=True`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix labels stored in model config for token classification examples
Author: sgugger
Pull request: yes

# What does this PR do?

This PR fixes the labels stored inside the config a model in our token classification example. Currently, when the dataset used comes from the dataset library, the correspondence stored is the identity (0->0, 1->1, etc) which is not super useful.

Fixes #15474 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
line 111 in convert_megatron_bert_checkpoint.py cause error "AttributeError: 'Namespace' object has no attribute 'get'"
Author: bugface
Pull request: no

The new script convert_megatron_bert_checkpoint.py for converting Megatron checkpoint to HF model has a minor error at line 111  

```
config.intermediate_size = ds_args.get("ffn_hidden_size", 4 * ds_args.hidden_size)
```

ds_args is <class 'argparse.Namespace'> which does not have "get" function


change to 

```
config.intermediate_size = ds_args.ffn_hidden_size if "ffn_hidden_size" in ds_args else 4 * ds_args.hidden_size
```
fix the issue.



Comments:
hey @bugface, thanks for opening an issue! Do you want to open a PR to fix this? :)
Number-specific tokenization changes
Author: avijit-thawani
Pull request: no
Labels: name: New model

# 🌟 New model addition

## Model description

I wanted to contribute a bunch of number-specific LMs proposed in recent work. Most of these are not architecture changes but simple tokenization tricks such as converting a number `329` to `3.29e2` (scientific; [Zhang et al. 2020](https://aclanthology.org/2020.findings-emnlp.439/)) or `3 2 9` (digit splitting; [Nogueira et al. 2021](https://arxiv.org/abs/2102.13019)) or `e2` (exponent only; [Spokoyny et al. 2020](https://aclanthology.org/2020.emnlp-main.385/) and [Thawani et al. 2021](https://aclanthology.org/2021.emnlp-main.557/)). The motivation is that several industrial applications require number-heavy NLP but struggle with existing models.

I discussed a specific way to do this on team slack with @SaulLu for, say, NumBERT (scientific notation) which involves adding a new model (tokenizer-only) and uploading the pretrained weights to the hub. I wanted to open a broader discussion here about more such number-tokenizer-only methods, some of which may not even have pretrained weights. The hope would be to make some abstract intervention (perhaps at the tokenizer level) to let the user configure GPT or BERT tokenizer as `number_tokenizer=exponent`. 

But perhaps clubbing methods from different papers into one model/tokenizer is against HF's philosophy? If so, I could proceed with trying to simply incorporate them as individual models - is it fine if some of them do not have pretrained weights available?

## Open source status

* [X] the model implementation is available: [T5 fine-tuned on arithmetic, based on HF transformers](https://github.com/castorini/transformers-arithmetic) and [NumBERT/scientific, based on google-bert original code](https://github.com/google-research/google-research/tree/master/numbert)
* [X] the model weights are available: [BERT pretrained with scientific notation](https://console.cloud.google.com/storage/browser/gresearch/numbert)
* [X] who are the authors: @spokoyny @XikunZhang @DeepakRamachandran @iftenney @yanaiela @rodrigonogueira4 @lintool 


Comments:
tokenizer truncation_side is not set up with from_pretrained call
Author: LSinev
Pull request: no

## Environment info

- `transformers` version: 4.16.1

### Who can help
Library:
- Tokenizers: @SaulLu
- Pipelines: @Narsil

## Information

Model I am using (Bert, XLNet ...): **GPT2Tokenizer**

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce
```
tokenizer = GPT2Tokenizer.from_pretrained("gpt2", truncation_side="left")
print(tokenizer.truncation_side)
```
```
right
```

## Expected behavior

```
left
```
## Possible solution

I believe the problem is in the missing part at `tokenization_utils_base.py` (just like the one for the padding side at https://github.com/huggingface/transformers/blob/master/src/transformers/tokenization_utils_base.py#L1457-L1462)
```
        # Truncation side is right by default and overridden in subclasses. If specified in the kwargs, it is changed.
        self.truncation_side = kwargs.pop("truncation_side", self.truncation_side)
        assert self.truncation_side in [
            "right",
            "left",
        ], f"Truncation side should be selected between 'right' and 'left', current value: {self.truncation_side}"
```
Feel free to add this solution if it is good enough without writing tests, as I don't have transformers dev environment ready for PR right now.


Comments:
Thanks for the detailed issue @LSinev ! Indeed, this seems to be a desirable change! 

@Narsil, it looks like we doubled the work, we should choose one of the 2 PRs. :relaxed: 

Haha no worries @SaulLu .

Whichever you prefer as a PR, I don't have a real preference for the test choice. (Yours has much more coverage, mine is much simpler)

Thank you! Personally, I think it would be good to test the behavior on as many tokenizer classes as possible, so we'll start with mine? 

I will of course put both of you as co-authors.

Go right ahead, I'll close my PR.
[T5] Ignore sentinel indices for unsupervised denoising / masking objective?
Author: ahoho
Pull request: no

The [docs](https://huggingface.co/transformers/model_doc/t5.html#training) state that the masked language modeling objective is simply

```
input_ids = tokenizer.encode('The <extra_id_0> walks in <extra_id_1> park', return_tensors='pt')
labels = tokenizer.encode('<extra_id_0> cute dog <extra_id_1> the <extra_id_2> 
', return_tensors='pt')
model(input_ids=input_ids, labels=labels)
```

I was wondering if I need to manually set the `additional_special_tokens_ids` (corresponding to the `<extra_id_#>` sentinels) in the `labels` to `-100` during training so that they are ignored by the loss, as I believe would be the case for the `[MASK]` tokens in BERT? It seems that at least the `pad_token_id` is ignored in [`examples/seq2seq`](https://github.com/huggingface/transformers/blob/a09fe140c1c059baf05c4f97e5b4e83c719608db/examples/seq2seq/finetune.py#L153), but it's not clear if this ought to be true for the sentinels as well. My suspicion is _no_, but since there's no canonical MLM code for T5, I figured it was worth checking.

(I asked this in the forums and in a somewhat related issue, but was recommended to post here & tag @patrickvonplaten / @thomwolf)

Comments:
Hey @ahoho - good question! 

I'm pretty confident that you should mask all sentinel tokens (with -100) and only compute the loss the "real" labels being "cute dog", "the"  and "
". 

Also they are definitely not automatically ignored as is done for the pad_token_id in `examples/seq2seq`

I could not find a more detailed explanation in the paper - so maybe @craffel could take a quick look as well and confirm (hope it's fine to tag you here Colin)

No need to treat the sentinel tokens specially (masking out their loss or otherwise). The model is trained to output both the sentinel tokens and the filled-in blanks.
MLflowCallback to log run_name argument
Author: HenryMaguire
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

When using the MLflowCallback (set as default for Trainer), I would like to log the `run_name` argument passed to TrainingArguments as the Run Name on the MLflow dashboard. Currently runs are being logged as nameless.
E.g. see below.
![Screenshot 2020-11-13 at 10 29 32](https://user-images.githubusercontent.com/2844109/99062592-21abd500-259b-11eb-9caf-c43d2bcb31dc.png)


## Motivation

Trainer makes training 🤗  models so easy and MLflow is great for organising experiments/caching artifacts. I would like to make it easier to organise experimental runs and make research easier, particularly for larger teams. This feature would be a very simple patch on the original PR #8016 

example usage
```
training_args = TrainingArguments(
    label_names=['labels_t1', 'labels_t2'],
    output_dir='./runs,          # output directory
    run_name='multitask_clf_<run_name>',
)
trainer = Trainer(
    model=model,        # the instantiated 🤗 Transformers model to be trained
    args=training_args,        # training arguments, defined above
    train_dataset=train_dataset,        # training dataset
)
```

## Your contribution

I can submit a PR


Comments:
Please don't hesitate to suggest a PR, `run_name` is there just for this reason!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


It will be useful to have this, and it will organize all the experiments better.

Is there any update on this issue and #12841? I have a very simple one-line solution of passing `args.run_name` (which currently serves for `wandb`) to `mlflow.start_run` that can fix this. I can submit a PR in case of need.

@HenryMaguire can you send link to your notebook or  colab?
XLM-RoBERTa model for QA seems not properly work
Author: antoniolanza1996
Pull request: no
Labels: name: wontfix

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.3.1 (and I've also tried installing Transformers from `master`, see details below)
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help
albert, bert, GPT2, XLM: @LysandreJik

## Information

Model I am using (Bert, XLNet ...): 
[deepset/xlm-roberta-large-squad2](https://huggingface.co/deepset/xlm-roberta-large-squad2) 

The problem arises when using:
* [x] the official example scripts: [run_squad.py](https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: **SQuAD 2.0 dev set evaluation**
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
```
! wget https://raw.githubusercontent.com/rajpurkar/SQuAD-explorer/master/dataset/dev-v2.0.json
! python transformers/examples/question-answering/run_squad.py \
  --model_type xlm-roberta \
  --model_name_or_path 'deepset/xlm-roberta-large-squad2' \
  --do_eval \
  --do_lower_case \
  --predict_file 'dev-v2.0.json' \
  --output_dir 'output' \
  --overwrite_output_dir \
  --version_2_with_negative
```

## Expected behavior
There are some values mismatch between:
1. values reported in the model card [here](https://huggingface.co/deepset/xlm-roberta-large-squad2#performance)
2. values obtained when Transformers is installed using `pip install transformers` 
3. values obtained when Transformers is installed from master

In particular:
- Reported metrics in the model card: `"exact": 79.45759285774446, "f1": 83.79259828925511`
- Transformers installed from pip: `'exact': 64.67615598416576, 'f1': 77.27580544355429`
- Transformers installed from master: `'exact': 60.11959090373114, 'f1': 76.13129575803934`


Comments:
Thanks for reporting, will investigate.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


hello, this command "  --model_type xlm-roberta \" not work for me,
can you help me? please
Isn't `transformers.utils.fx` compatible with torch 1.10+ ?
Author: hyunwoongko
Pull request: no

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/conda/lib/python3.7/site-packages/transformers/utils/fx.py", line 579, in symbolic_trace
    tracer = HFTracer(batch_size=batch_size, sequence_length=sequence_length, num_choices=num_choices)
  File "/opt/conda/lib/python3.7/site-packages/transformers/utils/fx.py", line 244, in __init__
    f"Found an incompatible version of torch. Found version {torch_version}, but only version "
ImportError: Found an incompatible version of torch. Found version 1.10.0, but only version 1.9 is supported.
```

torch 1.10+ is also providing fx, but why the Transformers limits it to torch 1.9?

cc @stas00 @thomasw21 @michaelbenayoun

Comments:
I found [this](https://github.com/huggingface/transformers/pull/14321).

cc @michaelbenayoun 

Yeah, it had to do with torch doing not back-compat changes in the FX API - since this is all experimental.

But yes, Michael is your man.

Since FX API is in flux still I think ideally we should target pt-1.11 which should be out in probably a month or so.

I think fx became stable as of torch 1.10, so we should maybe drop that constraint?

Hi, you are right #14321 solves your issue, and as @thomasw21 said, if FX indeed became stable at torch 1.10, we will relax that constraint to something more like 1.10+.
Should merge the PR soon!
fix TFMarianMTModel output
Author: ydshieh
Pull request: yes

# What does this PR do?

One line change:

Current `TFMarianMTModel` returns `decoder`'s `last_hidden_state` as `encoder_last_hidden_state`. This PR fixes it.

```
encoder_last_hidden_state=outputs.last_hidden_state,  # index 0 of encoder outputs
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._