noamaanMulla-03/transformers-issues huggingface dataset:
Roberta (and BERT) tokenization converts "do not" to "don't"
Author: davisyoshida
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Roberta

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Execute the following:
```python
import transformers

tokenizer = transformers.RobertaTokenizer.from_pretrained('roberta-base')
print(tokenizer.decode(tokenizer.encode('is not')))
print(tokenizer.decode(tokenizer.encode('do not')))
```

The output is
```

 don't


 is not

```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior
The detokenization should not incorrectly introduce a contraction.

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.8.0
- Platform: Linux-5.5.3-arch1-1-x86_64-with-glibc2.2.5
- Python version: 3.8.1
- PyTorch version (GPU?): not installed (NA)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


Comments:
For anyone coming across this issue, you can disable all such transformations in `decode` by passing `clean_up_tokenization_spaces=False`. However, I maintain that this decoding behavior is not a sensible default.

fixed with #4024
GPT2 generations with specific words
Author: prabalbansal
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
@patrickvonplaten 
Just curious to generate a text with the specified words.
Generate function has a parameter bad_words, instead of removing these, I want to include these in generations. I try to edit in src/transformers/modeling_utils.py line 1194; to 
next_token_logits[batch_idx, banned_tokens[batch_idx]] = -float("inf")
changed to :
next_token_logits[batch_idx, banned_tokens[batch_idx]] = 1
 But this didn't work. Is there is any way we can do this?








Comments:
These `next_token_logits[batch_idx, banned_tokens[batch_idx]]` are logits in the `range(-inf, inf)` not probabilities. You can try to set all other logits except your banned_tokens to `-float("inf")`
XLNetLMHeadModel: target mapping with num_predict > 1 and labels not working
Author: hannodje
Pull request: no

Hi,

While trying to fine tune the XLNetLMHeadModel, I ran into the following error message:

`  File "[...]/lib/python3.8/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "[...]/lib/python3.8/site-packages/transformers/modeling_xlnet.py", line 1062, in forward
    loss = loss_fct(logits.view(-1, logits.size(-1)), labels.view(-1))
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
`

while calling the forward function like this:


               outputs = xlnet(input, # shape = (8, 50)

                                 attention_mask=attention_mask, # shape = (8, 50)

                                 perm_mask=perm_mask, # shape = (8, 50, 50)

                                 target_mapping=target_mapping, # shape = (8, 2, 50)

                                 token_type_ids=sequence_ids, # shape = (8, 50)

                                 labels=target) # shape = (8, 2)


According to the [docs](https://huggingface.co/transformers/model_doc/xlnet.html#transformers.XLNetLMHeadModel) the labels shape is expected to be (batch_size, num_predict). However using this shape causes the above mentioned error.

The error does not appear if we set num_predict to 1 and set labels.shape = torch.size([8])
The error also does not appear if we omit the labels and manually compute the loss 

Comments:
Hi @hannodje,

sorry to answer so late. Could you provide a complete code sample that can reproduce the error? One in which you set define all the variables `input`, `attention_mask`, `perm_mask`, `target_mapping`, `sequence_ids` and `target` and which defines which XLNetLMHeadModel you use. Ideally, I can copy / paste the code sample in a script without having to add code of my own to reproduce the error :-)  It's kinda hard to reproduce the error otherwise. Thanks!
Weights from pretrained model not used in GPT2LMHeadModel
Author: ankit-chadha
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): gpt2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)


## To reproduce

Steps to reproduce the behavior:

1. trained gpt2 using run_language_modeling
2. using run_generation.py

`backend_1        | 04/25/2020 18:37:27 - INFO - transformers.modeling_utils -   Weights from pretrained model not u
sed in GPT2LMHeadModel: ['transformer.h.0.attn.masked_bias', 'transformer.h.1.attn.masked_bias', 'transformer.h.2.a
ttn.masked_bias', 'transformer.h.3.attn.masked_bias', 'transformer.h.4.attn.masked_bias', 'transformer.h.5.attn.mas
ked_bias', 'transformer.h.6.attn.masked_bias', 'transformer.h.7.attn.masked_bias', 'transformer.h.8.attn.masked_bia
s', 'transformer.h.9.attn.masked_bias', 'transformer.h.10.attn.masked_bias', 'transformer.h.11.attn.masked_bias']`





Comments:
@patrickvonplaten - could you help elaborate on this?

Yeah, we need to clean this logging. The weights are correctly loaded into the model as far as I know (we have integration tests for pretrained GPT2 models so the weights have to be correctly loaded). Just need to clean up the logging logic there.

Actually this logging info is fine. Those masked bias are saved values using the `register_buffer()` method and don't need to be loaded.
Why GPT2LMHeadModel's loss mismatches accuracy?
Author: lx-kika
Pull request: no

# ❓ Questions & Help
Hello,
I'm pretraining GPT2LMHeadModel on my own data from scratch, however I find that loss decreases sharply, and the evaluation doesn't match the loss anymore.

## Details
I used two different ways to evaluate the LM model:
1.input [1,2,3...] label [2,3,4...] this way will compare the output of whole sentence with label once
2.input [1] label [2]
   input [1,2] label [2,3] but we compare only the output of 2's position and label 3
   input [1,2,3] label [2,3,4] but we compare only the output of 3's position and label 4
   and so on (each time count only last position's accuracy)
   finally sum up them

In my opinion, these two ways should give the same result, however when loss decreases sharply, these two results don't match anymore. This happens only in the case that learning rate is high, like 0.001 or even bigger.

Could you help me explain how it happens?
![loss](https://user-images.githubusercontent.com/62126666/80560370-4089b600-8a13-11ea-9b7f-76ff158da59a.jpg)


Comments:
Hi @lx-kika, 

Can you post a code snippet that calculates the different loss types you mention? 
Also, I think it should be fine to just use the first method to calculate the loss - I don't really see a reason as why to use the second method, no?
Issue with non-text files and bertabs example
Author: db1981
Pull request: no

Hello team,
I've been running the bertabs example on some plain txt files using the latest version available from the repository (pulled on 4/28/2020). 

The folder where I placed the txt files for processing contains the configuration file automatically generated by the filesystem (".DS_store", as I'm running macos, but I suspect Windows would have similar issues). 

Because the configuration file is binary, the following piece of code in util_summarization.py (lines 56-59) fails with a UnicodeDecodeError exception:

with open(document_path, encoding="utf-8") as source:
   raw_story = source.read()
   story_lines, summary_lines = process_story(raw_story)
return document_name, story_lines, summary_lines

I added a "try ... except" block to handle the UnicodeDecodeError exception and return an empty raw_story string if the exception is raised. This fixed the issue for me.



Comments:
So the issue is fixed for you then?
BERT as encoder and a transformer as a decoder.
Author: ayanamongol
Pull request: no
Labels: name: Usage

Hi, there
Is there probability to build a model of the BERT as the encoder and the transformer as the decoder?

Thanks.

Comments:
The encoder-decoder or bart may be what you want.

I think you should take a look into the ``encoder-decoder`` framework: https://huggingface.co/transformers/model_doc/encoderdecoder.html

Note that currently only Bert2Bert is possible.
Error in Calculating Sentence Perplexity for GPT-2 model
Author: states786
Pull request: no

Hi,

I am using a following code to calculate the perplexity of sentences on my GPT-2 pretrained model:
       
```
       tokenizer = GPT2Tokenizer.from_pretrained('gpt-model')
       config = GPT2Config.from_pretrained('gpt-model')
       model = GPT2LMHeadModel.from_pretrained('gpt-model', config=config)
       model.eval()
       
  def calculatePerplexity(sentence,model,tokenizer):
        input_ids = torch.tensor(tokenizer.encode(sentence)).unsqueeze(0) 
        input_ids = input_ids.to('cpu')
        with torch.no_grad():
            outputs = model(input_ids, labels=input_ids)
        loss, logits = outputs[:2]
        return math.exp(loss)
```
For some of the sentences from my testing corpus, I am getting following error:

**Token indices sequence length is longer than the specified maximum sequence length for this model (1140 > 1024). Running this sequence through the model will result in indexing errors**

How can I resolve this error? Kindly advise.

Comments:
The longest input length a pretrained GPT2 model can treat depends on its `n_position` value. You can look it up here e.g. https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json . 
If you use a pretrained-model you sadly can only treat sequences <= 1024.

For you own model you can increase `n_position` and retrain the longer position encoding matrix this way. 

If you are just interested in the perplexity you could also simply cut the input_ids into smaller input_ids and average the loss over them. It will not exactly be the same, but a good approximation.

Is it being calculated in the same way for the evaluation of training on validation set?

Secondly, if we calculate perplexity of all the individual sentences from corpus "xyz" and take average perplexity of these sentences? will it be the same by calculating the perplexity of the whole corpus by using parameter "eval_data_file" in language model script?

1) Yes 

2) No -> since you don't take into account the probability `p(first_token_sentence_2 | last_token_sentence_1)`, but it will be a very good approximation.

Hope this answers your question
How to make run_language_modeling.py work for transformer-xl?
Author: bajajahsaas
Pull request: no

The code in [run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py) does Language modeling for any pre-trained model. However, for transformer-xl, we need additional arguments to the forward function like `mems` and even [run_transfo_xl.py](https://github.com/huggingface/transformers/blob/master/examples/contrib/run_transfo_xl.py) performs `model.reset_length()` operation using `mem_len` and `tgt_len` as arguments. 

I am not sure how these functions are performed in this generic code for runningLM for any pre-trained model. Any suggestions will be helpful.

Comments:
Yeah we don't have a run_language_modeling.py script yet. It's on a TODO list :-). Also see: #2008 .
If you feel like writing one, it would be great :-)
run_generation.py - use of < redirect for input text file only reads first line - solution requested
Author: GenTxt
Pull request: no

Example:

python3 examples/run_generation.py --model_type=gpt2 --model_name_or_path="models/custom-774M" --length=400 < test.txt > test_gpt2_400.txt

Version 2.0.3 and earlier accepts a simple linux redirect < test.txt and reads the file line-by-line while generating the corresponding output to > test_gpt2_400_out.txt

This works perfectly and does not require any changes. The test.txt is a simple 3 line text file with 1 sentence on each line.

Versions 2.0.4 to 2.0.8 do not accept the < redirect in the same manner. The above only reads the first line of 'test.txt', generates, then exits. Output is the same for the first line.

Is there a simple change to the 2.0.8 version of 'run_generation.py' that will return this useful terminal function under ubuntu 18.04?

Have tried 'while - read' loops and only managed to generate the Model prompt >>> 3x requesting terminal input:

exec 3<"test.txt"

while IFS= read -r line <&3; do

python3 examples/run_generation.py --model_type=gpt2 --model_name_or_path="models/custom-774M" --length=400 

done > test_gpt2_400_out.txt

Any help is appreciated.

Comments:
I think in your case it might make more sense to use the `TextGenerationPipeline`. It's a two liner that does the same as the script. You can wrap a `open(inpt_text_file)` and `file.write()` around these two lines in your use case. 

See documentation for the generation pipeline here: https://huggingface.co/transformers/usage.html#text-generation

Closing for now
XLNet Generation appears to reference padding text in run_generation script
Author: thesamuel
Pull request: no
Labels: name: Ex: Generation

When generating with XLNet in the `run_generation.py` script, the outputs seem to reference the context from the padding text. For instance, given the prompt "We propose a", XLNet generates "We propose a boy Go Ya Ya, a young Iriel Farg, to be named Rasputin."

This seems to reference the padding text:
https://github.com/huggingface/transformers/blob/0866669e751bef636fa693b704a28c1fea9a17f3/examples/text-generation/run_generation.py#L62-L71

From what I understand, this padding text should not influence the generation, since the padding ends with an end of sentence token. Is this behavior expected?

Full command I used for reference:
```bash
python -m examples.run_generation --model_type xlnet --model_name_or_path xlnet-base-cased --prompt "We propose a"
```

Comments:
Hi @thesamuel, 

Ideally, the padding text should not influence the outcome, but this is more a hack to make XLNet work with short prompts, than actual science.

Also note that it is recommended now to use the TextGeneration Pipeline instead of the `run_generation` script: 

```
from transformers import pipeline
generator = pipeline("text-generation", model="xlnet-base-cased")
print(generator("We propose a "))
```
**Note**: This works well for XLNet only after merging PR: #4686. So for pipelines to work, you either have to wait a bit or work on the branch of the PR: #4686. 

As a default the pipeline employs sampling instead of greedy search. You might also want to play around with the generation hyperparameters here a bit for better results. To learn more about how to effectively use the many parameters for text generation, you might want to take a look at: https://huggingface.co/blog/how-to-generate
[WIP] seq2seq example
Author: mgoldey
Pull request: yes

This PR presents an example seq2seq use case and bug fixes necessary for this to execute with reasonable accuracy. 

The utils_seq2seq.py file defines the data format for training data, and the run_seq2seq.py file takes training, development, and test data and produces a model. The README.md discusses how to execute this toy problem. The specific toy problem in use here is formatting a date string to the American style, which is a trivial example. On my local setup using GPUs, this example executes within 5 minutes. Production models should include more learnings.

I welcome feedback about how to strengthen performance here and the best route to increase testing.

This relies on a few bug fixes which have been incorporated in this branch
- Without a fix for #3038, PreTrainedEncoderDecoder won't instantiate at all. 
- Without a fix for #2435, BERT models fail completely on this use case as the BERT decoder isn't instantiated correctly without CrossAttention in that case.
- ~I strongly suspect that the input to the decoder in the PreTrainedEncoderDecoder class is incorrect as present in the code base, and commit https://github.com/huggingface/transformers/commit/9fcf73afbcfa18918234592039da7bd409820431 has a proposed fix. It doesn't make sense to have the expected token ids as input to the decoder when the decoder needs to learn how to decode from the embeddings.~ Incomplete understanding - will fix

Comments:
As a non-blocking question, I do note that a lot of the examples use argparse to parse comparatively long lists of arguments. I've maintained the extant style in this PR to avoid causing noise and confusion Would it be acceptable if I broke with this style to use a JSON file to store all the arguments for an experiment?

Hi @mgoldey, sorry for only responding now. Thanks a lot for adding a seq2seq example :-) I will take a look early next week and maybe we can have a quick chat how to merge this PR and https://github.com/huggingface/transformers/pull/3383. 



That sounds good. I'm still tweaking things on my end for accuracy and improved logic as I get more familiar with the code base here. I'll see if I can rebase of #3383 by then, depending on my other workload. Feel free to reach out via google hangouts if you're comfortable.

Sorry, to answer only now! I'll will soon add a Encoder-Decoder google colab that shows how to use seq2seq 

Thanks - fine to close. We've moved forward without using seq2seq due to poor overall accuracy with the scale of data in place.
Fix XLNet batch generation bug
Author: neonbjb
Pull request: yes

When doing batch generation with XLNet, only the first element in the batch contains any predictions. This seems to be caused by the target_mapping being improperly initialized in prepare_inputs_for_generation.

Comments:
Hi @neonbjb,

sorry to answer waaaay to late.
Batch generation currently does not work. See #3021

We are not sure yet when and how to add this feature.

Uhhhh.. but it could with this PR? I have it working on my cloned repo and used it in this writeup. Your call though.
https://nonint.com/2020/03/27/fine-tuning-xlnet-for-generation-tasks/
[Generation Beam Search] Fix bug when changing the <EOS> token for generate
Author: patrickvonplaten
Pull request: yes

This PR fixes https://github.com/huggingface/transformers/issues/4121 . When comparing int `!=` should be used here instead of `is not`. 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=h1) Report
> Merging [#4745](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/47a551d17b6ed2eaf03301f049006d559fca5cf3&el=desc) will **increase** coverage by `0.22%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4745/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4745      +/-   ##
==========================================
+ Coverage   77.14%   77.36%   +0.22%     
==========================================
  Files         128      128              
  Lines       21073    21073              
==========================================
+ Hits        16256    16304      +48     
+ Misses       4817     4769      -48     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4745/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.66% <ø> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4745/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.29% <ø> (+0.35%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4745/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.85% <0.00%> (+0.41%)` | :arrow_up: |
| [src/transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/transformers/pull/4745/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19ncHQyLnB5) | `86.21% <0.00%> (+14.10%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=footer). Last update [47a551d...491f4e2](https://codecov.io/gh/huggingface/transformers/pull/4745?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).