noamaanMulla-03/transformers-issues huggingface dataset:
Strange exception
Author: ghost
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help


Hi everybody, 
I'm keeping experiment with the following colab: 
https://colab.research.google.com/github/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb#scrollTo=X9_Go99fvW-z

however I got a new error after the fine-tuning starts:

Exception in thread Thread-20:
Traceback (most recent call last):
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/parallel_loader.py", line 141, in _loader_worker
    _, data = next(data_iter)
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 354, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 394, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "<ipython-input-6-cc30f67d4a07>", line 34, in collate_batch
    input_ids = torch.stack([example['input_ids'] for example in batch])
  File "<ipython-input-6-cc30f67d4a07>", line 34, in <listcomp>
    input_ids = torch.stack([example['input_ids'] for example in batch])
TypeError: string indices must be integers

Is this due to the new changes introduced yesterday for the DataCollate ?



Comments:
The notebook is not using the collation function from `transformers` but the `T2TDataCollator` it defines. Apart from stopping subclassing `DataCollator` (as it's not a class anymore on master) this shouldn't be a problem.

Hi, @antoniomastro1996 , if you are using transformers from master branch then there are few changes.
As @sgugger  said, `DataCollator` is not a class anymore, so just don't subclass. Also as datacollator is now a `callable`, rename the `batch_collate` method to  `__call__`.

Or you can use version 2.11.0 and it'll work as it is. I'll update the notebook once the new version is released. Let me know if you run into any other issues.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How can I use tokenizer.encode_plus to input and encode 2 sentences - (query,answer) pair for training a BERT binary classifier?
Author: soumya-ranjan-sahoo
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
Here my intention is to train a BERT binary classifier which classifies if an answer corresponding to a given query is correct. How do I proceed to encode the query, answer pair in the Input?

@soumya-ranjan-sahoo You could check the [token type ids](https://huggingface.co/transformers/glossary.html#token-type-ids). Please feed the question and answer to the, for instance, encode_plus function and generate type type ids. 

Then you could just feed the token type ids with, for instance, input ids and attention masks to conduct classification

@bright1993ff66 Great. I was successful in my experiment. But now I have a follow-up question. I understand the maximum length of the permissible words for BERT is 512. In my case (sentence pair classification) does that imply the combined word length for the query and the answer has to be 512 since I have huge answers for my experiment. 
Surprisingly I was able to fine-tune BERT with query and answer (most query-answer pair have a combined word length of more than 512), and BERT didn't throw any error or warnings. How did it fine-tune or what it exactly did with such sentences? 

Thank you. 



This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to  save the created embedding of the text corpus.
Author: PaulJohny
Pull request: no
Labels: name: wontfix

After converting the text corpus into word embedding how can we save the embeddings into a file. It is not practical and feasible to create word embedding each and every time. Is there ant work around to save the embedding. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
FillMaskPipeline return word-piece 
Author: orena1
Pull request: no
Labels: name: wontfix

# 🚀 . Feature request
Not sure exactly if this is a bug/feature request/ or just me not understanding correctly :).
I am trying to use the [FillMaskPipeline](https://github.com/huggingface/transformers/blob/master/src/transformers/pipelines.py#L739) but as far as I understand the pipeline returns a single word-piece for a \<mask\>. But the mask could be of a word or even a text-spans (for the case of BART).

Sample code:
```python
from transformers import pipeline
nlp =pipeline("fill-mask",model='bart-large')
nlp(f'Their expression was divine; and as they glanced at me timidly but with parted'
    f' lips in great <mask>, I forgot all thoughts of their conversion in feelings '
    f'that were far more earthly.')
#missing word is bewilderment, this is from librispeech

..
{'sequence': '
 Their expression was divine; and as they glanced at me timidly but with parted lips in great bewild, I forgot all thoughts of their conversion in feelings that were far more earthly.
',
  'score': 7.772801473038271e-05,
  'token': 33304},

print(tokenizer.decode(33304))
' bewild'
```
As can be seen above, one of the outputs is "bewild" which is the first word-piece in  bewilderment:
```python
[tokenizer.decode(i)+' ' for i in tokenizer.batch_encode_plus(['bewilderment'])['input_ids'][0]]
['
 ', ' bewild ', 'er ', 'ment ', '
 ']
```

## Your contribution

I assume we could go to the next output as see if it is a word-piece and if it is add it to the first token. not sure exactly if this is correct, specially when it seems that the size of the output is exactly as the number of input word-pieces.

Thanks


Comments:
I do not think generally there's a way of knowing that a given token is usually a sub-token vs. a complete "word"

Maybe a heuristic could be to do some kind of greedy decoding, iteratively adding a second `<mask>` after the first filled one and checking if the output score is above a certain threshold/above the previous one.

@julien-c @orena1 So knowing for one sub-token whether it is complete or not is on at the "token" level. What about a span text?

The closest I have found it https://github.com/huggingface/transformers/issues/3972

Hi @Diego999, I actually did not find a way to test/train span text on BART, although the paper mention that they train the model using span-text.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Lite transformer
Author: flozi00
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

## Model description
Transformer has become ubiquitous in natural language processing (e.g., machine translation, question answering); however, it requires enormous amount of computations to achieve high performance, which makes it not suitable for mobile applications that are tightly constrained by the hardware resources and battery. In this paper, we present an efficient mobile NLP architecture, Lite Transformer to facilitate deploying mobile NLP applications on edge devices. The key primitive is the Long-Short Range Attention (LSRA), where one group of heads specializes in the local context modeling (by convolution) while another group specializes in the long-distance relationship modeling (by attention). Such specialization brings consistent improvement over the vanilla transformer on three well-established language tasks: machine translation, abstractive summarization, and language modeling. Under constrained resources (500M/100M MACs), Lite Transformer outperforms transformer on WMT'14 English-French by 1.2/1.7 BLEU, respectively. Lite Transformer reduces the computation of transformer base model by 2.5x with 0.3 BLEU score degradation. Combining with pruning and quantization, we further compressed the model size of Lite Transformer by 18.2x. For language modeling, Lite Transformer achieves 1.8 lower perplexity than the transformer at around 500M MACs. Notably, Lite Transformer outperforms the AutoML-based Evolved Transformer by 0.5 higher BLEU for the mobile NLP setting without the costly architecture search that requires more than 250 GPU years. Code has been made available at https://github.com/mit-han-lab/lite-transformer
<!-- Important information -->

## Open source status

* [x] the model implementation is available: (give details)https://github.com/mit-han-lab/lite-transformer
* [x] the model weights are available: (give details) linked in readme of GitHub repo
* [x] who are the authors: (mention them, if possible by @gh-username)Zhanghao Wu • Zhijian Liu • Ji Lin • Yujun Lin • Song Han


Comments:
Very cool!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
After I resume learning, loss is greater than prev checkpoint
Author: urekalion
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help


## Details

I learned albert mlm model.
Often an error occurs (memory exception, ... )
so i resumed learning model from last checkpoint 
but, training loss goes back to the beginning.
what do i need to check ? 
thank,
![loss ](https://user-images.githubusercontent.com/4244158/84736498-ed7abb00-afe0-11ea-9b30-9e669e327241.png)




Comments:
Hi! What did you use to train your model? Was it the `run_language_modeling` script? Do you happen to have the command you used?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Compressive Transformer
Author: stvhuang
Pull request: no
Labels: name: wontfix

name: New model

# 🌟 New model addition

## Model description

<table>
    <tr><th>Title</th><td>Compressive Transformers for Long-Range Sequence Modelling (ICLR '20)</td></tr>
    <tr><th>arXiv</th><td><a href="https://arxiv.org/pdf/1911.05507.pdf">1911.05507</a></td></tr>
    <tr><th>Blog</th><td><a href="https://deepmind.com/blog/article/A_new_model_and_dataset_for_long-range_memory">A new model and dataset for long-range memory</a></td></tr>
</table>

__Compressive Transformer__ is an attentive sequence model which __compresses past memories__ for long-range sequence learning. The idea is similar to [Transformer-XL](https://arxiv.org/pdf/1901.02860.pdf). However, the memories are compressed in __Compressive Transformer__, making it leverage longer past memories compared to __Transformer-XL__.

## Open source status

- [ ] the model implementation is available
- [ ] the model weights are available
- [ ] who are the authors

Comments:
Interested in model weights too but currently not available. Author does mention releasing tf code here:

https://news.ycombinator.com/item?id=22290227

Requires tf 1.15+ and deepmind/sonnet ver 1.36. Link to python script here:

https://github.com/deepmind/sonnet/blob/cd5b5fa48e15e4d020f744968f5209949ebe750f/sonnet/python/modules/nets/transformer.py#L915

Have tried running as-is but doesn't appear to have options for training on custom data as per the paper and available data sets.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Train RobertaModel from scratch for my dataset
Author: raj5287
Pull request: no
Labels: name: wontfix

I am trying to train RobertaModel from scratch. I am following [this](https://huggingface.co/blog/how-to-train) blog but instead of `model = RobertaForMaskedLM(config=config)`, I am starting with `configuration = RobertaConfig()
model = RobertaModel(configuration)` and then continuing with other steps. But I am getting error `TypeError: forward() got an unexpected keyword argument 'labels'`. The whole code piece:
```
configuration = RobertaConfig()
model = RobertaModel(configuration)
from transformers import LineByLineTextDataset
dataset = LineByLineTextDataset(
    tokenizer=tokenizer,
    file_path="./train.txt",
    block_size=128,
)
from transformers import DataCollatorForLanguageModeling

data_collator = DataCollatorForLanguageModeling(
    tokenizer=tokenizer, mlm=False, mlm_probability=0.15
)
from transformers import Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir="./Model1",
    overwrite_output_dir=True,
    num_train_epochs=1,
    per_gpu_train_batch_size=64,
    save_steps=10_000,
    save_total_limit=2,
)

trainer = Trainer(
    model=model,
    args=training_args,
    data_collator=data_collator,
    train_dataset=dataset,
    prediction_loss_only=True,
)
trainer.train()
```
Is there some other way to do pre-training? Am i missing something here?

Comments:
Are you using transformers from source or from pip install ?
`labels` is introduced in a recent commit and available on master, if you  are on version 2.11.0 from pip install then use `lm_labels`

> Are you using transformers from source or from pip install ?
> `labels` is introduced in a recent commit and available on master, if you are on version 2.11.0 from pip install then use `lm_labels`

@patil-suraj  yes I am on version 2.11.0 so where should I use `lm_labels`

If you installed using pip then yes, use `lm_labels`

You'll need to change `DataCollatorForLanguageModeling`, also as you are using `Roberta` which means you training it for maksed language modelling, so you'll need set `mlm `to `True` 

but I am training a RobertaModel not a RobertaMaskedLM model.

What is your pre-training objective ? Roberta is pre-trained using masked language modelling objective 

i am training it from scratch for my own dataset. I want to use the vectors obtained from last layer for classification task

You can train `RobertaForMaskedLM` using the `mlm` objective and then load it in `RoberatForSequenceClassification` for classification .
`RoberatForSequenceClassification` will take care of taking last layer vector and feeding it to a classification layer.

yes I have done that now I want to compare it with others RF or GDBTs and extract some features and that's why I want to train RobertModel.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
[Benchmark] Jetson Nano DistillBERT SQuAD benchmark 
Author: arijitx
Pull request: no
Labels: name: wontfix

# 🖥 Benchmarking `transformers`

## Benchmark

DistillBERT SQuAD

## Set-up

- Jetson Nano
- Jetpack 4.4
- onnxruntime 1.13.1

Benchmark script https://gist.github.com/arijitx/1400d3d4e07fc517d6c5bfea506c2353

## Results

tokens/sec

|                    | Pytorch-GPU | Pytorch-CPU(4 cores) | onnx-CPU (4 core) | onnx-CUDA | onnx-TRT |
|--------------------|-------------|----------------------|-------------------|-----------|----------|
| Distill BERT SQuAD |         570 |                   61 |               107 |  605      |     fail     |



Comments:
For tensorrt you need to manage dynamic axis with a script in the nuphar folder of onnx runtime repo (`symbolic_shape_infer.py`).
It gives best result for small batch (like single example).
It requires some customization in parameters because default one are not that good (check the Nvidia doc for the workspace size, etc.)
Of course, it requires a specific compilation but I suppose you did it.
For CPU you may want to install the onnx runtime without gpu support so you get openmp.

Ya I did `symbolic_shape_infer.py` but still for some reason, the model is not running with tensorrtProvider, it starts consuming memory and then crashes after the memory is loaded, I did try with workspace_size , iteration as mentioned in the  [Onnx TensorRT Provider doc](https://github.com/microsoft/onnxruntime/blob/master/docs/execution_providers/TensorRT-ExecutionProvider.md) anything `ORT_TENSORRT_MAX_PARTITION_ITERATIONS  > 1 ` crashes. 

Still Jetpack 4.4 is not officially supported by onnx, and I couldn't find a way to export pytorch models with LongTensors directly to TensorRT engines :( 

Am able to run successfully in the CUDAProvider, CPU can be optimized further with a openmp specific build.

My own XP with onnx has been painful too :-) 
May be you will be interested in this article https://medium.com/@fanzongshaoxing/accelerate-pytorch-model-with-tensorrt-via-onnx-d5b5164b369 ?
It s about tensorrt on Python without onnx runtime (didn't try yet but was curious)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Help with Debugging TF Common tests
Author: sshleifer
Pull request: no
Labels: name: wontfix

I am a TF2 Noob trying to get TFBart working. Most tests pass besides the ones relying on save_pretrained and pt conversion. Has anybody experienced the following issues?
```
test_tf_compile_model:
h5py/h5o.pyx:202: in h5py.h5o.link
...
RuntimeError: Unable to create link (name already exists)
```
Or
```
test_pt_tf_model_equivalence:
AttributeError: tf_bart_model_9.tf_bart_encoder_9.tf_shared_embeddings_9.weight not found in PyTorch model
```

`transformers-cli env`:

```bash
- `transformers` version: 3.0.0
- Platform: Darwin-19.4.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Task specific params for pegasus-large to allow finetuning with correct generation_parameters
Author: sshleifer
Pull request: no

```
'task_specific_params': {'summ_xsum': {'max_length': 56, 'length_penalty': 0.8}}}
```

Comments:
Done!
```python
# Config values that vary between checkpoints: for testing and conversion
task_specific_params = {
    # These are task specific params for pegasus-large and normal params for finetuned checkpoints
    "summarization_xsum": {"length_penalty": 0.8, "max_length": 64, "max_position_embeddings": 512},
    "summarization_cnn_dailymail": {"length_penalty": 0.8, "max_length": 128, "max_position_embeddings": 1024},
    "summarization_newsroom": {"length_penalty": 0.8, "max_length": 128, "max_position_embeddings": 512},
    "summarization_wikihow": {"length_penalty": 0.6, "max_length": 256, "max_position_embeddings": 512},
    "summarization_multi_news": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_reddit_tifu": {"length_penalty": 0.6, "max_length": 128, "max_position_embeddings": 512},
    "summarization_big_patent": {"length_penalty": 0.7, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_arxiv": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_pubmed": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
    "summarization_gigaword": {"length_penalty": 0.6, "max_length": 32, "max_position_embeddings": 128},
    "summarization_aeslc": {"length_penalty": 0.6, "max_length": 32, "max_position_embeddings": 512},
    "summarization_billsum": {"length_penalty": 0.6, "max_length": 256, "max_position_embeddings": 1024},
    # this last entry is useless -- just for consistency
    "summarization_large": {"length_penalty": 0.8, "max_length": 256, "max_position_embeddings": 1024},
}
```
How to use encode_plus to force padding to specific length
Author: adriangrepo
Pull request: no

I am using the following code in the __getitem__() method of my dataset:

`    

    class MyDataset(Dataset):
        def __init__(self, myargs):

            #other code here
            self.tokenizer = BertTokenizer.from_pretrained('bert-base-cased)
            self.max_len =50

`

__getitem__ method:

`    

        def __getitem__(self, idx):
           #other code here
           text_encoded = self.tokenizer.encode_plus(
              text,
              add_special_tokens=True,
              padding=True,
              truncation=True,
              max_length=self.max_len,
              return_token_type_ids=False,
              pad_to_max_length=True,
              return_attention_mask=True,
              return_tensors='pt',)

        input_ids=text_encoded['input_ids'].flatten()
        attention_mask=text_encoded['attention_mask'].flatten()

    >>input_ids: torch.Size([9]), attention_mask: torch.Size([9])
    >>input_ids: torch.Size([21]), attention_mask: torch.Size([21])

`

Even though I have set padding and truncation to True and set a max_length the returned lengths of the input_ids and attention_mask values in the retured text_encoded dict are variable depending on the input text. Is this normal behaviour? if so how can I ensure that ecery rturned sample is padded out to and truncated at a specific length?

Comments:
I worked out that if you set padding=False then the text is padded out correctly with input_ids and attention_mask values of 0. This is the opposite setting of what I thought padding to mean but it works.

'''

     >>input_ids: torch.Size([1, 50]), attention_mask: torch.Size([1, 50]) #pre flatten

```
Multilingual MNLI model
Author: iuria21
Pull request: no
Labels: name: wontfix

Hi,

I'm trying Zero-Shot Learning  with 'facebook/bart-large-mnli' and it's pretty well, but I want to do it in Spanish. Is there any multilingual mnli model?

Thanks for your attention!

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Set the number of times to evaluate per epoch when using Trainer
Author: alexorona
Pull request: no
Labels: name: wontfix

Basically, I want to see how the training is going against the evaluation dataset twice per epoch when using `Trainer`. To calculate `logging_steps` so that `evaluate()` is called a certain number of times per epoch when training, will this hold-up to single and multi-device training?

```
evals_per_epoch = 2
logging_steps = len(train_dataset) / train_batch_size / gradient_accumulation_steps // evals_per_epoch
```

I'm getting myself confused over the implementation of `global_steps` and `logging_steps` inside of `Trainer.train`. 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.