noamaanMulla-03/transformers-issues huggingface dataset:
Report minimum system requirements for each architecture.
Author: ysig
Pull request: no
Labels: name: wontfix

Hi,

It would be nice if you wrote some minimum requirements in all of your models.
Like the size of the GPU (or the number of GPUs) for a batch size of 1 (or n_gpu).
This would make it really helpful for trying new staff.
For example what are the requirements of re-training or generating with CTRL?

Thanks in advance! 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Inconsistent tokenizer handling of max_len
Author: johncookds
Pull request: no

Hi, it seems that at least the RobertaTokenizerFast is not actually truncating encodings to the max_len when encoding(the same issue occurs with the other encoding functions). The BPE tokenizer from tokenizers does.

Below the problem is shown based on the 'how to train from scratch' example.

```
from tokenizers.implementations import ByteLevelBPETokenizer
from tokenizers.processors import BertProcessing
from transformers import RobertaTokenizerFast

good_tokenizer = ByteLevelBPETokenizer(
    "./BERT-Esperanto/vocab.json",
    "./BERT-Esperanto/merges.txt",
)
good_tokenizer._tokenizer.post_processor = BertProcessing(
    ("
", good_tokenizer.token_to_id("
")),
    ("
", good_tokenizer.token_to_id("
")),
)

good_tokenizer.enable_truncation(max_length=512)

bad_tokenizer = RobertaTokenizerFast.from_pretrained("./BERT-Esperanto/", max_len=512)

txt = "Mi estas Julien." * 1000

print(
    len(good_tokenizer.encode(txt).tokens),
    len(bad_tokenizer.encode(txt))
)

# results: 
512
5002
```

Comments:
Yes, similarly to the `good_tokenizer` where you enabled truncation, you should enable it for the `bad_tokenizer`:

```py
from transformers import RobertaTokenizerFast

actually_very_good_tokenizer = RobertaTokenizerFast.from_pretrained("./BERT-Esperanto/", max_len=512)

txt = "Mi estas Julien." * 1000

print(
    len(actually_very_good_tokenizer.encode(txt, truncation=True))
)

# 512
```

You can check the documentation [here](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.__call__).

Ah, gotcha my mistake, thanks for the quick response

Love the variable names here. (and the sample text) 🤣
grammar corrections and train data update
Author: DeepsMoseli
Pull request: yes
Labels: name: model card

- fixed grammar and spelling
- added an intro
- updated Training data references

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=h1) Report
> Merging [#5448](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d16e36c7e525aab4c08a6e60a7478e209498dc14&el=desc) will **decrease** coverage by `0.86%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5448/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5448      +/-   ##
==========================================
- Coverage   77.82%   76.95%   -0.87%     
==========================================
  Files         141      141              
  Lines       24608    24608              
==========================================
- Hits        19150    18938     -212     
- Misses       5458     5670     +212     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_mobilebert.py](https://codecov.io/gh/huggingface/transformers/pull/5448/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9tb2JpbGViZXJ0LnB5) | `23.62% <0.00%> (-73.11%)` | :arrow_down: |
| [src/transformers/generation\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5448/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9nZW5lcmF0aW9uX3RmX3V0aWxzLnB5) | `86.43% <0.00%> (+0.25%)` | :arrow_up: |
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5448/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `81.09% <0.00%> (+1.37%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/5448/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl9vcGVuYWkucHk=) | `94.98% <0.00%> (+74.19%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=footer). Last update [d16e36c...20f340c](https://codecov.io/gh/huggingface/transformers/pull/5448?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
licens
Author: TCUI22
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
Where did "prepare_for_model" go?  What is the replacement?
Author: ohmeow
Pull request: no

I'm working with already numericalized data (e.g., where the text has been converted to ids via `tokenizer.tokenize()`) and was using `prepare_for_model` to build the appropriate input dictionary ... ***but*** that method is gone in 3.0.

So ... what should I use/do now?

Thanks

Comments:
Hi! Why were you using `tokenize` + `prepare_for_model` instead of `encode`/`encode_plus` ? Let's see how to fit your use-case with the best approach!

Sure.

I'm the developer of [this library](https://ohmeow.github.io/blurr/) which integrates huggingface with fastai.  Probably the best thing is to look at the code [here](https://github.com/ohmeow/blurr/blob/master/blurr/data/core.py) to see what I'm doing.

The fastai bits don't work all that well with inputs composed of multiple tensors, and so my initial fastai transform converts the text to ids, which are then wrapped in a tensor to make fastai happy.  Before batches are created, I would use `prepare_for_model` to get the necessary transformer inputs (e.g. input_ids, attention_mask, etc...), pad to max length, etc..., using those ids.

@sgugger may have some thoughts on how to adapt my code better to v3 given he wrote most of those pieces in fastai :)

Thanks!

@LysandreJik Would it be possible that you link this issue in the "breaking changes" section for the 3.0.0 release 🤔 

In Flair we had the same issue :)

Hmm, I see, let me investigate. Does using the private `_prepare_for_model` solve your issue? I'll ping @n1t0 as well as he might have more info on that front.

@stefan-it, just did! Thank you.

Ok, indeed I'll add it as a breaking change also we could expose it publicly again in a 3.0.1 if it happens that many people were using it.

The main reason I made it private is that we don't have it in fast tokenizers (though we could maybe work on having it) and I'm trying to have both APIs come closer to each others.

@n1t0 do you think we could provide an implementation of this method in Fast tokenizers? It's basically all the post-processing (truncation + merging pairs + padding) after the conversion in integer indices.

We should be able to expose the post-processing for both `List[str]` and `List[int]` in `tokenizers`, but I'll have to check. I think the only problem in doing so is that all the mappings (chars <=> tokens <=> words) and offsets won't make any sense in this case.

Ok, we will release a patch to fix this breaking change (re-expose `prepare_for_models` for both slow and fast tokenizers with backward-compatible API) plus the one mentioned in #5377 probably tomorrow or early next week.
ValueError in T5 community colab notebook.
Author: y-rokutan
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): T5

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the community notebook https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb:
  * It raises subclassing error while defining `class T2TDataCollator(DataCollator)`. DataCollator is not a class anymore on the latest master branch, just `class T2TDataCollator()` fixes the error.
  * Even that bug is fixed, the notebook raises another ValueError while importing dataset.pt:
```
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/torch_xla/distributed/xla_multiprocessing.py", line 231, in _start_fn
    fn(gindex, *args)
  File "<ipython-input-18-4f8aea5d9d8b>", line 191, in _mp_fn
    main()
  File "<ipython-input-18-4f8aea5d9d8b>", line 145, in main
    train_dataset  = torch.load(data_args.train_file_path)
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 589, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 847, in _load
    result = unpickler.load()
  File "/usr/local/lib/python3.6/dist-packages/nlp/splits.py", line 493, in __setitem__
    raise ValueError("Cannot add elem. Use .add() instead.")
ValueError: Cannot add elem. Use .add() instead.
```

## To reproduce

Steps to reproduce the behavior:

1. Run the community colab notebook: https://github.com/patil-suraj/exploring-T5/blob/master/T5_on_TPU.ipynb
2. You will get the error.

## Expected behavior
* Not applicable

## Environment info
* It's running on colab.


Comments:
hi @y-rokutan, you are right, `DataCollator` is not a `Class` anymore, its a `callable`  now, so remove subclassing and change `collate_batch` method to ` __call__`  

I'll update the notebook once new version of transformers is released

hi @patil-suraj, thx for your quick reply. I appreciate your contributions.
Do you have any idea about ValueError issue? I'm googling for fix but still have the same error.

What is your nlp version ? Try using nlp==0.2.0

`!pip install -U nlp==0.2.0` worked! nlp0.3.0 seems not for this notebook. Thx!

hi i am running the notebook without any changes . But its very slow i think its not using TPU at all
Possible breaking undetected change to "data/processors/squad.py"
Author: Santosh-Gupta
Pull request: no

# ❓ Questions & Help

It looks like the squad data utils hasn't been updated to the new version.

https://github.com/huggingface/transformers/blob/fcf0652460753f8a81f7576e8abdaa6b3742f00e/src/transformers/data/processors/squad.py#L136

```
        encoded_dict = tokenizer.encode_plus(  # TODO(thom) update this logic
            truncated_query if tokenizer.padding_side == "right" else span_doc_tokens,
            span_doc_tokens if tokenizer.padding_side == "right" else truncated_query,
            truncation="only_second" if tokenizer.padding_side == "right" else "only_first",
            padding="max_length",
            max_length=max_seq_length,
            return_overflowing_tokens=True,
            stride=max_seq_length - doc_stride - len(truncated_query) - sequence_pair_added_tokens,
            return_token_type_ids=True,
        )
```

In squad utils, `encoded_dict` used to have a 'overflowing_tokens' key, but in the new version, all the overflowing tokens are returned as list of lists. But it looks like the data processor doesn't take this into account

```
        if "overflowing_tokens" not in encoded_dict or (
            "overflowing_tokens" in encoded_dict and len(encoded_dict["overflowing_tokens"]) == 0
        ):
            break
        span_doc_tokens = encoded_dict["overflowing_tokens"]
``` 

So this logic would interpret that there are no overflowing tokens since that key doesn't exist. 

The code I am working on is using version 2.11.0, doesn't have 'overflowing_tokens' either, but it doesn't have the list of lists like in version 3.0.0. It has something called 'overflow_to_sample_mapping', which returns only zeros, which I am not sure how to use. 

The 2.11.0 documentation doesn't mention this return value. (https://huggingface.co/transformers/v2.11.0/main_classes/tokenizer.html) . 

But it looks the data processor wouldn't give overflowing tokens or notice an issue for 2.11.0 as well. And it doesn't seem to have a method to get strided overflowing tokens. 

Comments:
Hi, overflowing tokens are handled differently between slow and fast tokenizers with fast tokenizers having better support.
Which kind of tokenizer are you using?

I am using the fast tokenizers 

Ok, currently we don't handle overflowing in fast tokenizers with this processing script.
This is on the short term roadmap though.
TF2 - how to access intermediate layers of pre-trained bert model?
Author: yagelardan
Pull request: no
Labels: name: wontfix

(I'm following [this](https://mccormickml.com/2019/05/14/BERT-word-embeddings-tutorial/)  pytorch tutorial about BERT word embeddings, and in the tutorial the author is access the intermediate layers of the BERT model.)

What I want is to acess the last, lets say, 4 last layers of a single token input to the BERT model in tensorflow2 code. Because each layer output a vector of length 764 - so last 4 layers will have a shape of 4*768=3072 (for each token).

How can I implement in TF/keras/TF2, to get the intermediate layers of pretrained model for a token input? (later I will try to get the tokens for each token in a sentence, but for now one token is enough).

I'm using the huggingface's BERT model:

!pip install transformers
from transformers import (TFBertModel, BertTokenizer)

bert_model = TFBertModel.from_pretrained("bert-base-uncased")  # Automatically loads the config
bert_tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
sentence_marked = "hello"
tokenized_text = bert_tokenizer.tokenize(sentence_marked)
indexed_tokens = bert_tokenizer.convert_tokens_to_ids(tokenized_text)

print (indexed_tokens)
>> prints [7592]
The output is a token ([7592]), which should be the input of the for the BERT model

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Don't discard entity_group when token is the last in the sequence.
Author: mfuntowicz
Pull request: yes

Signed-off-by: Morgan Funtowicz <funtowiczmo@gmail.com>

Comments:
LGTM!

Thanks @mfuntowicz !

Before:

```bash
In [6]: nlp("My name is Wolfgang and I live in Berlin")                                                                                                                                           
Out[6]: [{'entity_group': 'I-PER', 'score': 0.9991481900215149, 'word': 'Wolfgang'}]
````

With this PR:

```bash
In [5]: nlp("My name is Wolfgang and I live in Berlin")                                                                                                                                           
Out[5]: 
[{'entity_group': 'I-PER', 'score': 0.9991481900215149, 'word': 'Wolfgang'},
 {'entity_group': 'I-LOC', 'score': 0.9983668327331543, 'word': 'Berlin'}]
```

@LysandreJik CI error seems unrelated, is it ok for you if I merge?

Did you check this, @enzoampil? Just making sure to ping you as you contributed #3957 🤗

@julien-c Did a few checks as well and looks great! Was planning to include this in this PR #4987 (2nd point), but this seems to solve it cleanly already, so will consider this fix for that PR :smile:

UPDATE: Ended up modifying this fix in the PR above, due to cases where the last token was repeating (for the test cases set in the above PR).
Cannot Achieve Reproducability with Tensorflow Transformer Models
Author: ghost
Pull request: no
Labels: name: wontfix

I've been experimenting with the roberta-large model with both PyTorch and TensorFlow for a sentiment analysis task. With the PyTorch model, I am able to achieve 100% reproducability, however, this is not the case with the Tensorflow model.

I have set all the necessary seeds as follows:

```
seed_val = 3
os.environ['PYTHONHASHSEED'] = str(seed_val)
np.random.seed(seed_val)
random.seed(seed_val) 
tf.random.set_seed(seed_val)
```
and I am even using the fix described at https://github.com/NVIDIA/tensorflow-determinism:

```os.environ['TF_DETERMINISTIC_OPS'] = '1'```

I am not sure if reproducability is fully supported with the TensorFlow transformer models yet, or if I am doing something wrong.

Here is the link to the **CoLab notebook** containing my code:
https://drive.google.com/open?id=1xPTYPl8LyRrMgkiXUNtSbxxJ7pAslD2x

Thanks in advance

Comments:
I have the same issue and I get (sometimes wildly) different results from run to run. Here's my code:

https://github.com/dmitriydligach/Thyme/blob/master/RelKeras/et.py

Does anybody have a solution yet?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Toy size models versions for faster experimentation and testing
Author: obsh
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

To have a small(toy) in terms of number of parameters versions of models e.g.:
- `gpt2-toy`
- `bert-toy-uncased`
etc.
Alternative:  to have separate model type `toy-transformer`.

## Motivation

Working on the project and using transformers package our team noticed that it's difficult for some team members who had machines without GPU to experiment with code working with GPT2 model and to run tests for it. Also, it required significant resources on CI environment to run tests.

We've ended up by implementing a custom configuration for GPT2 model with a resulting model size ~10M parameters for local experimentation and testing.

I think it might be beneficial for the community to have tiny model versions in the package that weakens hardware requirements to start work and to run tests.
If it's unreasonable to implement tiny version for every model - having separate model type which implements same interfaces and have corresponding heads will also help to reach the same objective.

## Your contribution

If either of the options sounds reasonable I can participate in the implementation.


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
xlnet large 
Author: gogokre
Pull request: no
Labels: name: wontfix

If you train in batch 8 at max length = 512 with the xlnet-base-cased, you will learn well with changes in values such as Loss, Accuracy, Precision, Recall, and F1-score.

However, if you train with batch 2 at max length = 512 with a xlnet-large-cased, even if you run several times, there is almost no change, and precision is always 1.00000, and other values are only fixed.

Is it because the batch size is too small? Help.

[epoch1]
Tr Loss | Vld Acc | Vld Loss | Vld Prec | Vld Reca | Vld F1
0.39480 | 0.87500 | 0.39765 | 1.00000 | 0.87500 | 0.93333

[epoch2]
  Tr Loss | Vld Acc | Vld Loss | Vld Prec | Vld Reca | Vld F1
0.39772 | 0.87500 | 0.38215 | 1.00000 | 0.87500 | 0.93333

Comments:
Maybe the task is learning rate and batch size sensitive. You can try to decrease the learning rate when changing the batch size from 8 to 2. 

> Maybe the task is learning rate and batch size sensitive. You can try to decrease the learning rate when changing the batch size from 8 to 2.

Using xlnet-large-cased with batch size 2 and learning rate = 1e-5 is the same symptom. Is there another way?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Help needed with interpretation of the MLP class
Author: h56cho
Pull request: no
Labels: name: Help wanted

name: wontfix

name: Core: Modeling

Hello,

I am having some trouble understanding the MLP function used in the Hugging Face GPT-2, which is found [here](https://github.com/huggingface/transformers/blob/73028c5df0c28ca179fbe565482a9c2143787f61/src/transformers/modeling_gpt2.py#L204).

Q1. For MLP, why are we setting the n_state to be equal to 3072, which is 4 * n_embd?
Q2. Below is the definition for the MLP class:
```python
class MLP(nn.Module):
    def __init__(self, n_state, config):  # in MLP: n_state=3072 (4 * n_embd)
        super().__init__()
        nx = config.n_embd
        self.c_fc = Conv1D(n_state, nx)
        self.c_proj = Conv1D(nx, n_state)
        self.act = gelu_new
        self.dropout = nn.Dropout(config.resid_pdrop)
```
in the MLP definition above, what exactly do the lines ``` Conv1D(n_state, nx)``` (the object ```self.c_fc```), and ``` Conv1D(nx, n_state)```   (the object ```self.c_proj```) do?

Thank you,

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


h56cho: I'm not sure if you ask about the code or the algorithm.
As far as I understand from the code, the class MLP is a basic 2-layer neural network (2 consecutive conv1d + gelu activation). This will be used to construct a bigger [network](https://github.com/huggingface/transformers/blob/73028c5df0c28ca179fbe565482a9c2143787f61/src/transformers/modeling_gpt2.py#L215). I hope this (partially) answers your Q2 question.
Q1. For the code, I think the comment is related to the number 3072 mentioned in this [link](https://jalammar.github.io/illustrated-gpt2/)


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.