noamaanMulla-03/transformers-issues huggingface dataset:
batch_encode_plus not working for GPT2, OpenAI, TransfoXL when returning PyTorch tensors
Author: BramVanroy
Pull request: no

## 🐛 Bug

`batch_encode_plus` does not work on GPT2, OpenAI, and TransfoXL when returning PyTorch tensors. Note that the code does work when leaving out the `return_tensors` argument. In that case, the output of `encoded` looks normal.

## To Reproduce
```python
from transformers import *

TOKENIZERS = {
    'albert': (AlbertTokenizer, 'albert-base-v1'),
    'bert': (BertTokenizer, 'bert-base-uncased'),
    'distilbert': (DistilBertTokenizer, 'distilbert-base-uncased'),
    'gpt2': (GPT2Tokenizer, 'gpt2'),
    'openai': (OpenAIGPTTokenizer, 'openai-gpt'),
    'roberta': (RobertaTokenizer, 'roberta-base'),
    'transfoxl': (TransfoXLTokenizer, 'transfo-xl-wt103'),
    'xlm': (XLMTokenizer, 'xlm-mlm-enfr-1024'),
    'xlnet': (XLNetTokenizer, 'xlnet-base-cased')
}

text = ['I like bananas and cookies .',
        'You are not what I thought you were , though .',
        'Cookies are awesome .']

for tok_name, (tok_cls, tok_default) in TOKENIZERS.items():
    tokenizer = tok_cls.from_pretrained(tok_default)
    try:
        encoded = tokenizer.batch_encode_plus(text, return_tensors='pt')
    except Exception as e:
        print(f"{tok_name} failed: {e}")
```

Output on latest master:

```
gpt2 failed: Could not infer dtype of NoneType
openai failed: Could not infer dtype of NoneType
transfoxl failed: Could not infer dtype of NoneType
```

## Environment

* OS: Windows 10
* Python version: 3.7.3
* PyTorch version: 1.3
* PyTorch Transformers version (or branch): latest master
* Using GPU ? yes
* Distributed or parallel setup ? no


Comments:
The problem lies here

https://github.com/huggingface/transformers/blob/babd41e7fa07bdd764f8fe91c33469046ab7dbd1/src/transformers/tokenization_utils.py#L1003-L1006

since for these tokenizers `self.pad_token_id` is None.

Still having this issue running the above script :-(
Any ideas?

Env:
* OS: Windows 10
* Python version: 3.6.12
* PyTorch version: 1.5.0
* PyTorch Transformers version (or branch): transformers-4.5.1
* Using GPU ? yes
* Distributed or parallel setup ? no
Fixed trainer total_flos relaoding in distributed mode
Author: TevenLeScao
Pull request: yes

# What does this PR do?

There was a bug on the `total_flos` quantity when loading/reloading trainer states in distributed mode: when reloading a training, every process started from the total amount of floating-point operations. The next time they were aggregated, this caused the sum of the operations of all processes to be inflated. This PR fixes this behaviour by only storing a `current_flos` variable per process that comes back to zero every time it is logged, and keeping the total amount separate in the trainer state.

# Who to tag?

@sgugger 

Comments:
tensorflow model convert onnx
Author: Zjq9409
Pull request: no

I use this code to transfer TFBertModel to onnx:
”convert(framework="tf", model=model, tokenizer=tokenizer, output=Path("onnx_tf/segment.onnx"), opset=12)”
and the output log as follows:
`Using framework TensorFlow: 2.2.0, keras2onnx: 1.7.0
Found input input_ids with shape: {0: 'batch', 1: 'sequence'}
Found input token_type_ids with shape: {0: 'batch', 1: 'sequence'}
Found input attention_mask with shape: {0: 'batch', 1: 'sequence'}
Found output output_0 with shape: {0: 'batch'}
Found output output_1 with shape: {0: 'batch', 1: 'sequence'}
Found output output_2 with shape: {0: 'batch', 1: 'sequence'}
Found output output_3 with shape: {0: 'batch', 1: 'sequence'}
Found output output_4 with shape: {0: 'batch', 1: 'sequence'}
Found output output_5 with shape: {0: 'batch', 1: 'sequence'}
Found output output_6 with shape: {0: 'batch', 1: 'sequence'}
Found output output_7 with shape: {0: 'batch', 1: 'sequence'}
Found output output_8 with shape: {0: 'batch', 1: 'sequence'}
Found output output_9 with shape: {0: 'batch', 1: 'sequence'}
Found output output_10 with shape: {0: 'batch', 1: 'sequence'}
Found output output_11 with shape: {0: 'batch', 1: 'sequence'}
Found output output_12 with shape: {0: 'batch', 1: 'sequence'}
Found output output_13 with shape: {0: 'batch', 1: 'sequence'}`

but,I run the output onnx,an error is occured as follows:
 logits = session.run(None, inputs_onnx)
  File "/usr/lib64/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 124, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:token_type_ids:0

transformers           3.0.2

Comments:
and when I upgrade transformers version to 4.3.3, onnx version is 1.8.1,another error occurred:
Traceback (most recent call last):
  File "test_segment.py", line 104, in <module>
    session = onnxruntime.InferenceSession(output_model_path, sess_options, providers=['CPUExecutionProvider'])
  File "/usr/lib64/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 206, in __init__
    self._create_inference_session(providers, provider_options)
  File "/usr/lib64/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 226, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from onnx_tf/segment.onnx failed:Fatal error: BroadcastTo is not a registered function/op

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Bug in Hosted inference API
Author: hhou435
Pull request: no

Hello,I found that when I use the Hosted inference API, there will be some problems.
Some inference results show the entire sentence, but other inference results only show the mask token.
For example the model [uer/roberta-base-word-chinese-cluecorpussmall](https://huggingface.co/uer/roberta-base-word-chinese-cluecorpussmall).
When I input "中国的首都是北[MASK]",the results are:
![image](https://user-images.githubusercontent.com/59219579/109969085-72079800-7d2e-11eb-876a-2f66e0fd23b2.png)
![image](https://user-images.githubusercontent.com/59219579/109969434-dcb8d380-7d2e-11eb-9f15-84b3229f97cb.png)
Thanks


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Why the positional embeddings in bert are not inplemented by sin/cos as the original paper said? Are these embeddings trainable?
Author: SeaEagleI
Pull request: no

https://github.com/huggingface/transformers/blob/948b730f9777174335812cf76de2a9dd9e4cf20e/src/transformers/models/bert/modeling_bert.py#L172

Comments:
BERT uses absolute position embeddings by default. The sin/cos embeddings are from the original Transformer paper IRRC. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Error in run_squad.py with BartForQuestionAnswering model
Author: nrjvarshney
Pull request: no

I am using the  BartForQuestionAnswering model and getting the following error during evaluation 
`Evaluating:   0%|          | 0/315 [00:10<?, ?it/s]
Traceback (most recent call last):
  File "../run_squad.py", line 831, in <module>
    main()
  File "../run_squad.py", line 820, in main
    result = evaluate(args, model, tokenizer, prefix=global_step)
  File "../run_squad.py", line 325, in evaluate
    output = [to_list(output[i]) for output in outputs.to_tuple()]
  File "../run_squad.py", line 325, in <listcomp>
    output = [to_list(output[i]) for output in outputs.to_tuple()]
  File "../run_squad.py", line 73, in to_list
    return tensor.detach().cpu().tolist()
AttributeError: 'tuple' object has no attribute 'detach'`

It works perfectly fine with the bert-base-uncased model but fails in case of BART model.
What could resolve this issue?
Thanks

Comments:
Hello! Could you provide all the information required in the issue template?

Thank you! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
'Trainer' object has no attribute 'log_metrics'
Author: fatihbeyhan
Pull request: no

I was trying to use run_mlm.py to fine-tune roberta-large with a custom dataset on Google Colab.

!python /content/transformers/examples/language-modeling/run_mlm.py \
    --model_name_or_path roberta-large \
    --train_file /content/traincorpus.txt \
    --validation_file /content/devcorpus.txt \
    --do_train \
    --do_eval \
    --per_device_train_batch_size 2 \
    --output_dir finetunedmodel \
    --overwrite_output_dir True

I am able to train the model however at the end I can not see the evaluation due to an error:

Traceback (most recent call last):
  File "/content/transformers/examples/language-modeling/run_mlm.py", line 442, in <module>
    main()
  File "/content/transformers/examples/language-modeling/run_mlm.py", line 416, in main
    trainer.log_metrics("train", metrics)
AttributeError: 'Trainer' object has no attribute 'log_metrics'

There is nothing extra on colab env. I wasn't getting this error a week ago or something.

Comments:
Hi, this is a duplicate, see #10446 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fine tuning a pipeline
Author: thiziri
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
[Already asked](https://github.com/huggingface/transformers/issues/8127)

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->
I would like to fine-tune a pipeline using a model (pre-trained or fine-tuned).


Comments:
Hi! I'm sorry, I don't understand what you mean exactly. A pipeline uses a model and a tokenizer under the hood, do you mean you want to fine-tune the model used underneath, and use that fine-tuned model?

If that is so, it is simple: any model/tokenizer can be loaded in the pipeline via local path/hub identifier. I invite you to read the [documentation of the pipelines](https://huggingface.co/transformers/main_classes/pipelines.html#transformers.pipeline), especially the `model` and `tokenizer` arguments.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Fine tune of speaker embeddings model
Author: karthikgali
Pull request: no

# 🚀 Feature request

Provide a way to fine tune the X-vector speaker embeddings model using our own custom dataset. (https://huggingface.co/hbredin/SpeakerEmbedding-XVectorMFCC-VoxCeleb)

## Motivation

This will help in finetuning the model for new domain/speakers.



Comments:
Hi! I think this is more of a question for `pyannote` rather than for `transformers`?

cc'ing @hbredin for visibility :)

Thanks @julien-c for the ping.

@karthikgali please open an issue or discussion in [pyannote.audio Github repo](https://github.com/pyannote/pyannote-audio) instead.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
pytorch Albert quantization error
Author: Zjq9409
Pull request: no

I use  huggingface transformers 'albert_chinese_base',but in pytorch quantization,The following problem occurred:
File "test_simple.py", line 186, in <module>
    model_pt_quantized(input_ids=model_inputs["input_ids"], token_type_ids=model_inputs["token_type_ids"], attention_mask=model_inputs["attention_mask"])
  File "/work/runtime/torch/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/runtime/torch/lib/python3.6/site-packages/transformers/modeling_albert.py", line 563, in forward
    output_hidden_states=output_hidden_states,
  File "/work/runtime/torch/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/runtime/torch/lib/python3.6/site-packages/transformers/modeling_albert.py", line 346, in forward
    output_hidden_states,
  File "/work/runtime/torch/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/runtime/torch/lib/python3.6/site-packages/transformers/modeling_albert.py", line 299, in forward
    layer_output = albert_layer(hidden_states, attention_mask, head_mask[layer_index], output_attentions)
  File "/work/runtime/torch/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/runtime/torch/lib/python3.6/site-packages/transformers/modeling_albert.py", line 277, in forward
    attention_output = self.attention(hidden_states, attention_mask, head_mask, output_attentions)
  File "/work/runtime/torch/lib64/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/work/runtime/torch/lib/python3.6/site-packages/transformers/modeling_albert.py", line 251, in forward
    self.dense.weight.t()
AttributeError: 'function' object has no attribute 't'

Comments:
I also meet the bug,I find that self.dense has been through prune_linear_layer() and return a nn.Linear as it should be,
```
>>> m = torch.nn.Linear(1,2)
>>> m.weight.t
<built-in method t of Parameter object at 0x7fcb7c3a45f0>
>>> m.weight.t()
tensor([[-0.0714,  0.7815]], grad_fn=<TBackward>)
```
but I can not find a way to fix it
If the maintainer answer your question,@me pls,thanks a lot!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
changing the way checkpoint is done in the new release 
Author: dorost1234
Pull request: no

Hi
Currently HuggingFace library checks the path for the latest checkpoint and then starts the training from there, this approach is not working due to following reason:
- lets assume you train the model with limit of 1 checkpoint and then you checkpoint only when the best eval accuracy is achieved, then if library loads the model from the last checkpoint, this is not training from the place training is stopped, and this can goes back a lot in time, since the last checkpoint is not necessarily  the last model we need to load from, and this is only the last model with best accuracy. 

To solve the issue, in addition to the checkpoint folders, one needs to introduce a "save_path_folder", and then when resuming the model trainining, loading from this path and not from last checkpoint folder anymore.

Please let me know if any part is not clear. thanks 

Comments:
I am not sure if I understand this correctly, how would the `Trainer` know if the last model is different from the actual last checkpoint, and if the model is only saved when the best eval accuracy reached, how would you save the last model to `save_path_folder`?

Also if you are saving the last model (and optimizer/scheduler) to some other directory, you could always pass that path as the `model_name_or_path` and change the `--output_dir` so the `Trainer` won't load the last checkpoint 

cc @sgugger 

> lets assume you train the model with limit of 1 checkpoint and then you checkpoint only when the best eval accuracy is achieved, then if library loads the model from the last checkpoint, this is not training from the place training is stopped, and this can goes back a lot in time, since the last checkpoint is not necessarily the last model we need to load from, and this is only the last model with best accuracy.

Yes, you should not use a `save_total_limit` of 1 in conjunction with metric tracking (such as `load_bst_model_at_end=True`) or accept you won't be able to resume training from the last checkpoint.  If you want to restart from scratch, you should just pass `--overwrite_output_dir`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
TypeError: can only concatenate str (not "int") to str
Author: omerarshad
Pull request: no

While running run_seq2seq.py for summarization task on my own CSV,  i get following error:


All the weights of BartForConditionalGeneration were initialized from the model checkpoint at sshleifer/distilbart-cnn-12-6.
If your task is similar to the task the model of the checkpoint was trained on, you can already use BartForConditionalGeneration for predictions without further training.
Traceback (most recent call last):
  File "/content/transformers/examples/seq2seq/run_seq2seq.py", line 645, in <module>
    main()
  File "/content/transformers/examples/seq2seq/run_seq2seq.py", line 476, in main
    load_from_cache_file=not data_args.overwrite_cache,
  File "/usr/local/lib/python3.7/dist-packages/datasets/arrow_dataset.py", line 1120, in map
    update_data = does_function_return_dict(test_inputs, test_indices)
  File "/usr/local/lib/python3.7/dist-packages/datasets/arrow_dataset.py", line 1091, in does_function_return_dict
    function(*fn_args, indices, **fn_kwargs) if with_indices else function(*fn_args, **fn_kwargs)
  File "/content/transformers/examples/seq2seq/run_seq2seq.py", line 448, in preprocess_function
    inputs = [prefix + inp for inp in inputs]
  File "/content/transformers/examples/seq2seq/run_seq2seq.py", line 448, in <listcomp>
    inputs = [prefix + inp for inp in inputs]
TypeError: can only concatenate str (not "int") to str



Comments:
Hi,

if  the `text_column` and `summary_column` arguments are not specified when running the script, it is assumed that the first column in a csv file contains the full text and the second column the corresponding summaries. From the error message, it seems your csv file has integers in the first column. 

done with this but now this error:


  File "run_seq2seq.py", line 645, in <module>
    main()
  File "run_seq2seq.py", line 518, in main
    pad_to_multiple_of=8 if training_args.fp16 else None,
TypeError: __init__() got an unexpected keyword argument 'model'

This is because the `DataCollatorForSeq2Seq` now adds a new `model` argument. Upgrading to master will fix this issue.

Also, as is mentioned at the very beginning of the [examples README,](https://github.com/huggingface/transformers/tree/master/examples#important-note) running the examples requires an install from source.



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
Option to output "test_preds_seq2seq.txt" text file with each checkpoint generated in "run_seq2seq.py"
Author: kingpalethe
Pull request: no

I had previously raised an issue in a mistaken belief that this functionality _used_ to exist in Transformers.

Current behavior: The  "test_preds_seq2seq.txt" file is created once, at the end of the last epoch.

For many Seq2Seq tasks, at least for mine, it would be very useful to get these predictions at each checkpoint, to see how the model changes over time as it is trained. 

thanks

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.