noamaanMulla-03/transformers-issues huggingface dataset:
Training T5-large model for Question Answering
Author: dulanafdo
Pull request: no
Labels: name: wontfix

Are there are any specific documents that I can follow, to do the training of the t5 model for Question answering?
I found this (https://huggingface.co/transformers/custom_datasets.html#qa-squad) on your website and it is not allowing me to use t5 model instead of DistilBert.

Comments:
This notebook should help: https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Adding call back to measure time of each step 
Author: cameronalonso2
Pull request: no
Labels: name: wontfix

Hi
I need to measure the time of each training step, with distributed version of seq2seq_trainer with TPUs, could you provide me with some hints how to measure the time? thanks 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
No loss in model output for TFElectraForPreTraining
Author: alibi123
Pull request: no
Labels: name: wontfix

No loss in model output for TFElectraForPreTraining, even though labels are provided.
This is from docs for ELECTRA:
`loss (optional, returned when labels is provided, tf.Tensor of shape (1,)) – Total loss of the ELECTRA objective.`

The problem is that I want to get loss and logits, but model outputs only logits.

My code snippet:
```
>>> import numpy as np
>>> from transformers import ElectraTokenizer, TFElectraForPreTraining
>>>
>>> model = TFElectraForPreTraining.from_pretrained('google/electra-base-discriminator')
All model checkpoint layers were used when initializing TFElectraForPreTraining.

All the layers of TFElectraForPreTraining were initialized from the model checkpoint at google/electra-base-discriminator.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFElectraForPreTraining for predictions without further training.
>>> tokenizer = ElectraTokenizer.from_pretrained('google/electra-base-discriminator')
>>> inputs = tokenizer('simple phrase', return_tensors='tf')
>>> labels = np.array([0] * 4)
>>> out = model(inputs, labels=labels)
>>> len(out)
1
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Measuring time when using xla_spawn on multiple cores 
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi,
I would like to measure the time of the program, when this is launched in distributed way on mutiple TPU cores with xla_spawn, I tried to write some callback like this which retunrs the same results with num_cores = 1 and 8, I am not sure how to handle it when this is distributed? I want to make sure pytorch XLA is using all cores indeed, thank you for your help. 

```
class PrinterTimeCallback(TrainerCallback):
  
    #A bare :class:`~transformers.TrainerCallback` that just prints the logs.
   
    def on_epoch_begin(self, args, state, control, **kwargs):
        """
        Event called at the beginning of an epoch.
        """
        if state.is_local_process_zero:
            self.start_time = time.time()


    def on_epoch_end(self, args, state, control, **kwargs):
        """
        Event called at the end of an epoch.
        """
        if state.is_local_process_zero:
            self.end_time = time.time()
            total_time = self.end_time - self.start_time
            print("total_time ", total_time)

```
   


Comments:
Hi, I think you should take a look at the [troubleshooting section over at pytorch/xla](https://github.com/pytorch/xla/blob/master/TROUBLESHOOTING.md), it has a lot of information on how to debug on TPU.

The metrics report they mention in the first paragraph can be output by the trainer with the `--debug` flag.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Disable default sigmoid function for single label classification Inference API
Author: Jiaxin-Pei
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Allow people to disable the default sigmoid function in TextClassificationPipeline (maybe via model cards?).

## Motivation

When we use the sequence classification model (e.g. RobertaForSequenceClassification) for regression tasks, the output may have different ranges other than [0,1], it would be better to allow configurations for the sigmoid function in TextClassificationPipeline.



Comments:
Hi! Right now the sigmoid function is applied when the pipeline detects that there is a single label. You would like the option to disable the sigmoid function in that case?

@LysandreJik 
Sorry for my late reply.
Yes, cuz when people are doing regression tasks using the single-label SequenceClassification model, the output range depends on the specific task. For example, when predicting age from the text, [0,1] output after a sigmoid function is not a good fit here. 

Indeed, I understand! I'm adding an option to return the raw outputs in #8328 

Thank you! I'm also wondering if this will be reflected by the Inference API? The inference API is using the sequence classification pipeline, therefore the API output on my model page is different from the original model output, which might confuse potential users. 

@Jiaxin-Pei see discussion in #8328

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Arrow file is too large when saving vector data
Author: weiwangorg
Pull request: no

I computed the sentence embedding of each sentence of bookcorpus data using bert base and saved them to disk. I used 20M sentences and the obtained arrow file is about 59GB while the original text file is only about 1.3GB. Are there any ways to reduce the size of the arrow file?

Comments:
I think it's expected because with `bert-base` each token will be embedded as a 768-dimensional vector. So if an example has n tokens then the size of embedding will be `n*768` and these are all 32-bits floating-point numbers.

Yes. I use datasets and I think this is a question about datasets, how to save vector data in a compressed format to reduce the size of the file. So I close this issue.
Need clarification in /examples/research_projects/rag/use_own_knowledge_dataset.py
Author: mayank31398
Pull request: no

Please explain the difference between:
'facebook/dpr-ctx_encoder-single-nq-base' and 'facebook/dpr-ctx_encoder-multiset-base'

Which datasets are the 2 models trained on?

Comments:
Indeed, the models have no model card. Maybe @lhoestq can help you out! 

Hi !
- 'facebook/dpr-ctx_encoder-single-nq-base' is the DPR context encoder model trained on NQ alone
- 'facebook/dpr-ctx_encoder-multiset-base' is the DPR context encoder model trained on the multiset/hybrid dataset defined in the paper. It includes Natural Questions, TriviaQA, WebQuestions and CuratedTREC

Thanks for clarifying @lhoestq
[s2s] Trainer vs PTL timings
Author: sshleifer
Pull request: no
Labels: name: wontfix

For the following two commands,

+ PTL finishes: 2.01 it/s, ~3H, 21.32 Rouge
+ Trainer: 1.0 it/s, roughly 5.5H, 21.36 Rouge

I wanted to report this so I don't lose track of it. Looked at the code, and don't see any obvious issue, besides that the slowdown is suspiciously close to 2x.



Any idea @patil-suraj ?

### PTL Command

```bash

export BS=32
export GAS=1

python finetune.py \
    --learning_rate=3e-5 \
    --fp16 \
    --gpus 1 \
    --do_train \
    --do_predict \
    --val_check_interval 0.25 \
    --n_val 500 \
    --num_train_epochs 2 \
    --freeze_encoder --freeze_embeds --data_dir cnn_dm \
    --max_target_length 142 --val_max_target_length=142 \
    --train_batch_size=$BS --eval_batch_size=$BS --gradient_accumulation_steps=$GAS \
    --model_name_or_path sshleifer/student_cnn_12_6 \
    --tokenizer_name facebook/bart-large \
    --warmup_steps 500 \
    --output_dir distilbart-cnn-12-6

```


### Trainer command
same as `builtin_trainer/train_distilbart_cnn.sh`:

```bash
export BS=32
export GAS=1
export m=sshleifer/student_cnn_12_6
export tok=facebook/bart-large
export MAX_TGT_LEN=142

python finetune_trainer.py \
    --model_name_or_path $m --tokenizer_name $tok \
    --data_dir cnn_dm \
    --output_dir distilbart-cnn-12-6-trainer --overwrite_output_dir \
    --learning_rate=3e-5 --sortish-sampler \
    --warmup_steps 500 \
    --fp16 \
    --n_val 500 \
    --gradient_accumulation_steps=$GAS \
    --per_device_train_batch_size=$BS --per_device_eval_batch_size=$BS \
    --freeze_encoder --freeze_embeds \
    --num_train_epochs=2 \
    --save_steps 3000 --eval_steps 3000 \
    --logging_first_step \
    --max_target_length 142 --val_max_target_length $MAX_TGT_LEN --test_max_target_length $MAX_TGT_LEN \
    --do_train --do_eval --do_predict --evaluate_during_training \
    --predict_with_generate --sortish_sampler
```

Comments:
I might be misreading progress bars. Running another test, will reopen if I can replicate. 

PTL (device 1) using less GPU ram it seems:

![image](https://user-images.githubusercontent.com/6045025/97605462-08285c00-19e5-11eb-9bc9-fd7900cde6b6.png)


Progress bars (note that PTL/Bottom is per epoch):

![image](https://user-images.githubusercontent.com/6045025/97605841-838a0d80-19e5-11eb-9e2a-eb71722f7308.png)


I'm also experiencing slow down on TPU's, didn't run the new changes on GPU yet. I"ll investigate this

Thx!

I've confirmed that builtin ~2x slower on 1 GPU than PTL. Same commands as above on a different machine. All the screenshots above are valid.

These seem to run at the same speed if you pass `--fp16_opt_level=O1` to pytorch-lightning. Verifying now and will post results in 5 hrs.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
TensorFlow Longformer model as a saved model with attention outputs
Author: LysandreJik
Pull request: no
Labels: name: wontfix

The TensorFlow implementation of the Longformer model has an issue with using the `tf.saved_model.save` API alongside the `output_attentions=True` configuration attribute. 

The test is skipped currently due to this issue.

Comments:
Don't manage to get this test passing even with the new design of #7562 -> the problem to me is that the shape of `attentions` in Longformer depends on the input tensor => so not sure we'll find a good solution here

If it can't pass the test defined in the common tests, then the best would be to override the test in the `LongformerModelTester` and do a test to ensure that the correct behavior still works, even if not adhering to the common tests.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
finetune_trainer being really slow on TPU
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi
I need to run finetune_trainer for a large scale project on TPU, here is the command I use 

`python xla_spawn.py --num_cores 8 finetune_trainer.py     --learning_rate=3e-5         --n_val 1000  --model_name_or_path t5-small --data_dir data/wmt_en_ro/ --output_dir /home/rabeeh/temp/ --overwrite_output_dir  --tpu_num_cores=8 --max_source_length=64 --max_target_length=64 --per_device_train_batch_size=32 --per_device_eval_batch_size=32 --label_smoothing=0.1 --task="translation" --logging_steps=200 --eval_steps=500 --num_train_epochs=1 --save_steps=500 --max_source_length=128  --max_target_length=128 --val_max_target_length=128 --test_max_target_length=128
`

It is running slow for me, and I was wondering if I can make it faster, can I make it run distributedly on multiple host machines? I really appreciate providing me with command to run this code efficiently on TPUs on cluster. Is there any setting I am missing in my command? 

thank you. 

Comments:
could you give me some numbers how much time it could take for training WMT on TPUs so I could compare? I appreciate letting me know how to do the setting on TPUs @patil-suraj 

AFAIK, there is a recent slowdown on TPU that @patil-suraj is investigating. 

I simple speedup would be to use `--model_name_or_path Helsinki-NLP/opus-mt-en-ro`


Hi Sam. I need to use T5 and I really got a very slow performance. could you tell me also the batch_size_per_host for v2-8 machine? since I also got SEGFAULT when using batch_size of 32 even per host. I really need to make this work for a project and I really need help. I appreciate assisting me with setting things up on TPU. this is really very slow and I do not know what I can do about it. thanks 

Hi Sam, 
Is there any chance to set a 10 minutes time with me, if I could ask some questions on setting up things on TPUs? I really do not have access to ask anyone questions on this and I need help to set things up. I greatly appreciate it thanks 

could you tell me the type of instance I need to create? maybe I am making mistake with using cloud? thanks

Sorry for interrupting.

I'm sorry if I'm wrong, but I'm thinking that how about using `--tpu_num_cores` instead of `--num_cores`.

I'm a complete newbie to TPU and I'm currently seeking a way to use of TPU in the Transformer Trainer and I found this issue.
Currently, the `TrainingArguments` class seems to use `tpu_num_cores` as an argument. You may be able to use `tpu_num_cores` instead of `num_cores`.
I'm trying to figure out what `tpu_num_cores` has to do with `num_cores`.

https://github.com/huggingface/transformers/blob/master/src/transformers/training_args.py#L290

I'm sorry if I'm misguided.

Excuse me, you are using `xla_spawn.py`, and it passes its `--num_cores` as `--tpu_num_cores` to `finetune_trainer.py`.
I apologize for commenting without fully understanding the situation.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I encountered an error when I was running code that the object could not be called.But the BertTokenizer doesn't exist in my code.
Author: YeNiTing
Pull request: no
Labels: name: wontfix

## Error details
     
- `transformers` version:2.11.0

Traceback (most recent call last):
File "main.py", line 108, in
fire.Fire()
File "D:\software\Anaconda3\envs\ynt2\lib\site-packages\fire\core.py", line 138, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "D:\software\Anaconda3\envs\ynt2\lib\site-packages\fire\core.py", line 468, in _Fire
target=component.name)
File "D:\software\Anaconda3\envs\ynt2\lib\site-packages\fire\core.py", line 672, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "main.py", line 69, in train
output = model(x)
File "D:\software\Anaconda3\envs\ynt2\lib\site-packages\torch\nn\modules\module.py", line 550, in call
result = self.forward(*input, **kwargs)
File "C:\Users\YNT\Desktop\重要文件夹\数据集\EMLo相关\pytorch_bert_elmo_example-master\model.py", line 40, in forward
word_embs = self.get_bert(x)
File "C:\Users\YNT\Desktop\重要文件夹\数据集\EMLo相关\pytorch_bert_elmo_example-master\model.py", line 90, in get_bert
ids = self.tokenizer(sentence_lists, padding=True, return_tensors="pt")
TypeError: 'BertTokenizer' object is not callable


Comments:
Hi, this is probably due to a mismatch of your transformers version.

Please complete the template, otherwise we cannot help you.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Running seq2seq_trainer with iterable datasets 
Author: rabeehkarimimahabadi
Pull request: no
Labels: name: wontfix

Hi
Looking into https://github.com/huggingface/transformers/blob/master/examples/seq2seq/seq2seq_trainer.py line 117, if the dataset is iterative, then the code does not return a distributed sampler so in case one wants to train on TPUs on multiple cores with iterable datasets, I was wondering if you could assist me how I can use the codes for iterable datasets on TPUs
thanks

Comments:
Maybe @patil-suraj has an idea!

Hi
I really appreciate assisting me with this question, I am still struggling with it, could you tell me some pointers, how to add the handling capability of iterative datasets to finetune_trainer.py codes. thanks  @patil-suraj 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
ETA on TFEncoderDecoderModel and is BERTShare from https://arxiv.org/pdf/1907.12461.pdf planned?
Author: anicolson
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Is there a plan for BERTShare from https://arxiv.org/pdf/1907.12461.pdf to be an option for the EncoderDecoderModel?

Also, I can see that an TFEncoderDecoderModel is on the 'ToDo' list for the [EncoderDecoder Framework](https://github.com/huggingface/transformers/projects/23). Any chance of an expected time of completion of this would be greatly appreciated. 

## Motivation

Having an easy to use seq2seq model integrated into hugging face (with TensorFlow) would help my research immensely. Also, models like BERTShare are much more parameter efficient.

## Your contribution

I am happy to help in any form. Not sure where help is needed tbh.


Comments:
I think we can keep this open, this looks like a fun project. Pinging @patrickvonplaten to let him know!

The models of https://arxiv.org/pdf/1907.12461.pdf  are already added. You can check them out here (they are not called shared, but are shared indeed): https://huggingface.co/models?search=google%2Froberta2roberta

Also, I'll be releasing an in-detail notebook about these models on Monday, so stay tuned :-) 

No ETA on TFEncoderDecoder models, but it's definitely on the roadmap :-) 

> The models of https://arxiv.org/pdf/1907.12461.pdf are already added. You can check them out here (they are not called shared, but are shared indeed): https://huggingface.co/models?search=google%2Froberta2roberta
> 
> Also, I'll be releasing an in-detail notebook about these models on Monday, so stay tuned :-)
> 
> No ETA on TFEncoderDecoder models, but it's definitely on the roadmap :-)

Thanks, I am switching from TF to PyTorch :)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Electra multi-gpu pretraining.
Author: 652994331
Pull request: no
Labels: name: wontfix

Hi, I am pretraining the Electra model with my own data, for now, I am pretraining using one GPU in my machine. Can we use multi GPUs to pretrain Electra? Thanks for your reply


Comments:
That would depend of the training script. All of our models are `nn.Module`s so all can be trained on multi-GPU, it just depends on the script. What script are you using?

@LysandreJik  Hi, thanks for your reply, i am not pretty sure about script, what I am doing now is using TensorFlow google Electra, but it seems I haven't figure out how to use multi gpu in TensorFlow, that's why i am here, Do u have any advice, I mean maybe I can just use Electra in this project to achieve multi-gpu pretraining?

Yes, you could check [this thread](https://discuss.huggingface.co/t/electra-training-reimplementation-and-discussion/1004) and use their project to train ELECTRA, which is based on this repository.

@LysandreJik  thanks, however, according to this issue https://github.com/richarddwang/electra_pytorch/issues/5 , I guess they haven't figure out how to use multigpu yet,

Ah, then I don't think I can help you further, unfortunately.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[s2s] label smoothing loss should be normalized
Author: sshleifer
Pull request: no
Labels: name: Help wanted

name: wontfix

by the number of padding tokens in a batch.
Currently, if you change `--train_batch_size` or `--max_target_length`, your loss value will scale wildly, making it hard to compare runs.

Comments:
Hi, I'm interested in taking a look at this. Could you please point out where to start?

1) On a branch try to add some logic to `label_smoothed_nll_loss` https://github.com/huggingface/transformers/blob/master/examples/seq2seq/utils.py#L35 such that changing `train_batch_size` doesn't wildly change loss.
2) validate (or ask for help validating) that the change does not hurt fine-tuning performance.

The existing code is copied from `fairseq`, so we need to be fairly sure that the change does no harm before we merge it.

cc @patil-suraj


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Summarization outputs on T5-small gets truncated 
Author: harung1993
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to the Hugging Face forum: https://discuss.huggingface.co/ .
     You can also try Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. In this case, make sure to tag your
     question with the right deep learning framework as well as the
     huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     -->

## Details
<!--  --> I been fine tuning t5-small for my own dataset but every time I set a max_length it just truncates the output. For example my input statement is : 

**When I first entered high school I was very nervous as it was a new school for me and it was a big adjustment. I was overwhelmed with work and mentally wasn't staying optimistic as I found it hard to manage my time and make friends. I felt like I wasn't good enough, and this caused me to treat myself like I wasn't worthy of being at such a place. In terms of behavior to others, I would say it made me more shy while still adapting to the new environment.** 

and my output is as follows:

 **when I first entered high school I was very nervous as it was a new school for me and it was a** 

My generate is as follows: 
**( input,
                                min_length= 0,
                                max_length=25,
                                length_penalty=2.0,
                                num_beams=4,
                                early_stopping=True )**

Is it possible for me to make it not truncate at the end? and also make it generate a reasonable summary ?

<!-- You should first ask your question on the forum or SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on the forum/Stack Overflow**:

Comments:
Hey @harung1993, 

sorry I'm having trouble understanding your question here. Also this seems like a question that should rather be posted in https://discuss.huggingface.co/ . We are trying to use github issues only for bug reports. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.