output padding different to zero in embedding layer

Core: Modeling

Internals of the library; Models.

Usage

General questions about the library

# 🐛 Bug
On embedding layer, the token corresponding to the padding does not return 0.
## Information

Model I am using (Bert):

Language I am using the model on (English):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
import torch
from transformers.tokenization_bert import BertTokenizer
from transformers.modeling_bert import BertEmbeddings

tokenizer = BertTokenizer.from_pretrained('bert-base-cased')

class Config:
    vocab_size = tokenizer.vocab_size
    hidden_size = 768
    max_position_embeddings = 512
    type_vocab_size = 2
    hidden_dropout_prob = 0.1
    layer_norm_eps = 1e-12

max_length = 10
sentence = "I eat a green apple"
tokens = tokenizer.encode(sentence)
tokens += [tokenizer.pad_token_id] * (max_length - len(tokens))
print(tokens)

embedding = BertEmbeddings(Config)
input_ids = torch.tensor([tokens])
emb = embedding(input_ids)

print(emb[0][-1])  # the last step should return 0 tensor
```
## Expected behavior
I expect to get a zero output tensor

## Environment info
- `transformers` version: 2.3.0
- Platform: Ubuntu
- Python version: 3.7
- PyTorch version (GPU?): 1.3.1 (CPU)
- Tensorflow version (GPU?): no
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no


fp16 is not compatible with the current activation code when pytorch is less than 1.4.0

[gelu = getattr(F, "gelu", _gelu_python)](https://github.com/huggingface/transformers/blob/53ce3854a16ad2a715bc6ac8af3e30c18b5a1d11/src/transformers/activations.py#L21)

should be changed to:

```python
if torch.__version__ < '1.4.0':
    gelu = _gelu_python
else:
    gelu = getattr(F, "gelu", _gelu_python)
```


Create README.md for xlnet_large_squad

model card

Related to pretrained model cards


pipeline("sentiment-analysis")() can't handle more than 2 sentences

Core: Pipeline

Internals of the library; Pipeline.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): pipeline("sentiment-analysis")

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
>>> from transformers import pipeline
>>> analyzer = pipeline('sentiment-analysis')
Downloading: 100%|██████████████████████████████| 230/230 [00:00<00:00, 146kB/s]
>>> analyzer(["OK"]*10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python3.6/site-packages/transformers/pipelines.py", line 490, in __call__
    scores = np.exp(outputs) / np.exp(outputs).sum(-1)
ValueError: operands could not be broadcast together with shapes (10,2) (10,)
>>>
```

`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Getting 10 results

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.0
- Platform: ubuntu 19.04
- Python version: 3.6
- PyTorch version (GPU?): 1.4.0 GPU
- Tensorflow version (GPU?): 1.14.0 GPU
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


BERT model breaks during FP16 Apex training on the latest update (2.5.0) - due to gelu function

PyTorch

Anything PyTorch

Should Fix

This has been identified as a bug and should be fixed.

# 🐛 Bug

BERT breaks during FP16 training due to the gelu function.

```
File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 407, in forward

hidden_states, attention_mask, head_mask[i], encoder_hidden_states, encoder_attention_mask

File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__

result = self.forward(*input, **kwargs)

File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 379, in forward

intermediate_output = self.intermediate(attention_output)

File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in __call__

result = self.forward(*input, **kwargs)

File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/transformers/modeling_bert.py", line 332, in forward

hidden_states = self.intermediate_act_fn(hidden_states)

File "/home/user/miniconda/envs/py36/lib/python3.6/site-packages/torch/nn/functional.py", line 1125, in gelu

return torch._C._nn.gelu(input)

RuntimeError: "GeluCUDAKernelImpl" not implemented for 'Half'
```

## Information

The reason this is happening is because in `modeling_bert.py`, in 2.4.1 we had:
```
def gelu(x):
    """ Original Implementation of the gelu activation function in Google Bert repo when initially created.
        For information: OpenAI GPT's gelu is slightly different (and gives slightly different results):
        0.5 * x * (1 + torch.tanh(math.sqrt(2 / math.pi) * (x + 0.044715 * torch.pow(x, 3))))
        Also see https://arxiv.org/abs/1606.08415
    """
    return x * 0.5 * (1.0 + torch.erf(x / math.sqrt(2.0)))

...

ACT2FN = {"gelu": gelu, "relu": torch.nn.functional.relu, "swish": swish, "gelu_new": gelu_new, "mish": mish}
```
whereas in 2.5.0 we have:
```
ACT2FN = {"gelu": gelu, "relu": torch.nn.functional.relu, "swish": swish, "gelu_new": gelu_new, "mish": mish}
```
where `gelu` now comes from `activations.py` as:
`gelu = getattr(F, "gelu", _gelu_python)` on line 21.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform: Linux
- Python version: 3.6
- PyTorch version (GPU?): 1.2.0 CUDA 10.0
- Tensorflow version (GPU?): 2.5.0
- Using GPU in script?: V100
- Using distributed or parallel set-up in script?: No, but using Apex FP16 training


Add standardized get_vocab method to tokenizers

This PR adds a `get_vocab` method to the `PretrainedTokenizers` to standardize extracting vocabularies from tokenizers.

Comments:
- I didn't do anything with fast tokenizers. cc'ing @mfuntowicz for his thoughts there.
- I opted to keep it a method rather than a `@property` in order to encourage users to primarily use existing methods like `convert_tokens_to_ids` for general encoding/decoding purposes and use `get_vocab` only when they need the entire vocabulary.
- For tokenizers which rely on `sentencepiece`, I was unable to figure out a better way to get the vocabs than to loop through it. If someone knows a better way, please let me know.


OpenAIGPTDoubleHeadsModel throws CUDA OOM with large number of candidates

Core: Modeling

Internals of the library; Models.

# 🐛 Bug

I am trying to train the `OpenAIGPTDoubleHeadsModel`. I find that large number of candidates can cause CUDA OOM errors.

Case 1 (single training example with 67 candidates): CUDA OOM
```
input_ids.shape: torch.Size([1, 67, 275])
mc_token_ids.shape: torch.Size([1, 67])
lm_labels.shape: torch.Size([1, 67, 275])
mc_labels.shape: torch.Size([1])
token_type_ids.shape: torch.Size([1, 67, 275])
```

Case 2 (single training example with 3 candidates): works fine!
```
input_ids.shape: torch.Size([1, 3, 275])
mc_token_ids.shape: torch.Size([1, 3])
lm_labels.shape: torch.Size([1, 3, 275])
mc_labels.shape: torch.Size([1])
token_type_ids.shape: torch.Size([1, 3, 275])
```

## Information

Model I am using: `OpenAIGPTDoubleHeadsModel`

Language I am using the model on: English

The problem arises when using my own modified scripts based on the `transfer-learning-conv-ai` repo by Hugging Face.

## To reproduce
Simply try training `OpenAIGPTDoubleHeadsModel` with larger number of candidates (such as 67).

## Expected behavior

3 or 67 candidates shouldn't matter, both cases 1 and 2 should work fine without CUDA OOM.

## Environment info

- `transformers` version: 2.3.0
- Platform: Amazon Linux (Deep Learning AMI)
- Python version: 3.6
- PyTorch version (GPU?): the one shipped with the pytorch_p36 conda env in Amazon DL AMI
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes and No


Small fix: default args for torch-lightning

Fix to the default argument passing to torch-lightning.


New tokenizers issue in NER demo

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

When I use transformers 2.5 I get the following error when running the run_ner demo. It seems to work from when I use 2.4. I am guessing this is because of a slight difference in the tokenization script? It seems to fail even when running with the base German run.sh in the ner/ directory.

```Traceback (most recent call last):
  File "run_pl_ner.py", line 233, in <module>
    trainer = generic_train(model, args)
  File "/content/transformers/examples/ner/transformer_base.py", line 268, in generic_train
    trainer.fit(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/trainer.py", line 911, in fit
    self.single_gpu_train(model)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/trainer/distrib_parts.py", line 464, in single_gpu_train
    self.optimizers, self.lr_schedulers = self.init_optimizers(model.configure_optimizers())
  File "/content/transformers/examples/ner/transformer_base.py", line 92, in configure_optimizers
    * float(self.hparams.num_train_epochs)
  File "/usr/local/lib/python3.6/dist-packages/pytorch_lightning/core/decorators.py", line 19, in _get_data_loader
    value = fn(self)  # Lazy evaluation, done only once.
  File "/content/transformers/examples/ner/transformer_base.py", line 132, in train_dataloader
    return self.load_dataset("train", self.hparams.train_batch_size)
  File "run_pl_ner.py", line 50, in load_dataset
    dataset = self.load_and_cache_examples(labels, self.pad_token_label_id, mode)
  File "run_pl_ner.py", line 175, in load_and_cache_examples
    pad_token_label_id=pad_token_label_id,
  File "/content/transformers/examples/ner/utils_ner.py", line 182, in convert_examples_to_features
    assert len(label_ids) == max_seq_length
```


Optimized squad.py multi-threading

If there are few examples, don't bother multi-threading . The base cost of multi-threading in Python is expensive. I can't upload a picture of the call-stack visual profile because of my stupid company firewall, but multi-threading in python depends on serializing each object before sending it off. That process has a 5 second overhead coming from <method 'dump' of '_pickle.Pickler' objects>. Don't take my word for it - test it yourself. This small optimization reduces invocation run-time from 6s down to 1.1s for a single example inference where the len(examples) == 1. This optimization singular but it should be echoed across all pipelines in the Transformers repo.

ps: I also see a lot of lists with appends across the Transformers repository as a whole. This is a big speed suck. Look into using collections.deques more - deques are like lists but are highly optimized for appends.

Good Luck HuggingFace - you guys rock!


Update README.md

model card

Related to pretrained model cards

- I added an example using the model with pipelines to show that we have set```{"use_fast": False}``` in the tokenizer for Q&A as noticed in [issue](https://github.com/huggingface/transformers/issues/2920)
- I added a Colab to play with the model and pipelines
- I added a Colab to discover Huggingface pipelines at the end of the document


Fix for fast tokenizers save_pretrained compatibility with Python.

The name of generated file doesn't match between tokenizers and transformers tokenizers, so transformers is not able to load model saved with tokenizers models.


[WIP] Add a trainer tool class to make the TF2 model training easier

Hello,

I decided to open a cleaner PR because the other was a bit too messy in my opinion. Here the features that this Trainer class will be able to handle:

- [x] Single / Multiple GPU training. Distributed is across GPUs in the same host. Distribution across multiple machines will be for a future version.
- [ ] The training can be configured with a JSON file.
- [x] Handle multiple data processor to be able to train a model over different datasets.
- [x] Select and configure a specific loss/optimizer for a training
- [x] Create multiple checkpoints during the training in order to make it fault-tolerant.
- [x] Create the logs to be able to visualize the training in Tensorboard
- [x] The final model is saved in Hugging face transformer format and in TF saved model
- [x] Able to give a data directory where to find the datasets
- [x] Automatically handle dataset/model caching
- [ ] Run an evaluation over a test dataset with proper printed results such as the one proposed by the `seqeval` package

Currently the trainer class can be used over glue and xnli datasets with the available examples `examples/run_tf_xnli_with_trainer.py` and `examples/run_tf_glue_with_trainer.py`. I will add new examples for differents tasks and datasets.

The list of features above will be checked as things progress.

Ping @julien-c @LysandreJik @thomwolf : Do not hesitate to make proposals if you have new ideas of features or advices on a better implementation of this trainer.


Fix spell: EsperBERTo, not EspertBERTo

This misspelling almost drove me crazy :)


Add local_files_only parameter to pretrained items

closes https://github.com/huggingface/transformers/issues/2867

Setting local_files_only=True disables outgoing traffic:
- etags are not looked up
- files are not downloaded (config, tokenizer, model)

An appropriate error is thrown when this argument may be the cause why a model cannot be loaded.

```python
import pyinstrument
from transformers import DistilBertConfig, DistilBertModel, DistilBertTokenizer

class TreeProfiler():
    def __init__(self, show_all=False):
        self.profiler = pyinstrument.Profiler()
        self.show_all = show_all # verbose output of pyinstrument profiler

    def __enter__(self):
        print("WITH TREE_PROFILER:")
        self.profiler.start()

    def __exit__(self, *args):
        self.profiler.stop()
        print(self.profiler.output_text(unicode=True, color=True, show_all=self.show_all))

def main():
    with TreeProfiler(show_all=True):
        config = DistilBertConfig.from_pretrained('distilbert-base-uncased', local_files_only=True)
        model = DistilBertModel.from_pretrained('distilbert-base-uncased', local_files_only=True)
        tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased', local_files_only=True)

if __name__ == '__main__':
    main()
```

The above snippet will throw an error message when the expected files are not present in the cache.  When they are, though, everything is loaded fine without the need of any additional lookups.


Getting the same results when evaluating Model2Model with different encoder inputs.

After fine-tuning the Model2Model with 'bert-base-uncased', I am getting the same losses values, no matter what is the encoder input. On the PreTrainedEncoderDecoder documentation, it's said that "During prediction, we perform one forward pass through the encoder,
          and then perform several forward passes with the encoder's hidden
          state through the decoder to decode a full sequence."
I couldn't find the place on the source code, that does the connection between the encoder and the decoder. If someone could help me to show me where this is happening, it will be a great help,
thanks!


Make RobertaForMaskedLM implementation identical to fairseq

Core: Modeling

Internals of the library; Models.

closes https://github.com/huggingface/transformers/issues/1874

The implementation of RoBERTa in `transformers` differs from the original implementation in [fairseq](https://github.com/pytorch/fairseq/tree/master/fairseq/models/roberta), as results showed (cf. https://github.com/huggingface/transformers/issues/1874). I have documented my findings here https://github.com/huggingface/transformers/issues/1874#issuecomment-588359143 and made the corresponding changes accordingly in this PR.

Someone should check, however, that removing `get_output_embeddings()` does not have any adverse side-effects.

In addition, someone who is knowledgeable about Tensorflow should check the TF implementation of RoBERTa, too.


What does ## mean in the bert vocab?

Core: Tokenization

Internals of the library; Tokenization.

What does ## mean in the bert vocab?

some words are starts with ##, such as ##a ##m ##er ##h ，I don't quite understand。


Masked LM implementation details

Core: Modeling

Internals of the library; Models.

TensorFlow

Anything TensorFlow

I read the source code of `TFBertMLMHead`, it seems that, this layer just predict the whole sequence, rather than predict the `MASKED` tokens.

`TFBertMLMHead` just do these things:

* transform the `hidden state` from the last encoder layer
* `predictions = tf.matmul(hidden_state, input_embedding_matrix)`, with shape (batch_size, sequence_length, vocab_size)
* return the `predictions` to calculate loss

The inputs are simply the `hidden state` of the last encoder layer.

But the implemenmtation from [google-research/bert](https://github.com/google-research/bert/blob/cc7051dc592802f501e8a6f71f8fb3cf9de95dc9/run_pretraining.py#L240) needs extra inputs `masked_lm_positions` and `masked_lm_weights`, and then use these inputs to calculate the masked lm loss.

So, does the `TFBertMLMHead` miss something?


DistilRoberta Model fine tuning on Squad dataset

Distillation

Related to model distillation

PyTorch

Anything PyTorch

# 🐛 Bug

I am trying to train the **distilroberta-base** model on **SQuAD** dataset using distillation code. I have trained the Roberta model on SQuAD 2.0 dataset, so that I can use it as a teacher model. I am using the **distilroberta-base** model as student.

## Information

Model I am using : Roberta

Language I am using the model on: English

The problem arises when using:

* [x] my own modified scripts: (give details below)
I am using the run_squad_w_distillation.py code, with the following modification
- Added the imports relevant to roberta
- Removed token_type_ids from inputs while sending to student model

```python
MODEL_CLASSES = {
    "bert": (BertConfig, BertForQuestionAnswering, BertTokenizer),
    "xlnet": (XLNetConfig, XLNetForQuestionAnswering, XLNetTokenizer),
    "xlm": (XLMConfig, XLMForQuestionAnswering, XLMTokenizer),
    "distilbert": (DistilBertConfig, DistilBertForQuestionAnswering, DistilBertTokenizer),
    "roberta": (RobertaConfig, RobertaForQuestionAnswering, RobertaTokenizer)
}

```

```python
if args.model_type in ["roberta"]:
     del inputs["token_type_ids"]
```
The tasks I am working on is:
* [x] an official GLUE/SQUaD task: SQuAD 2.0

## To reproduce

Steps to reproduce the behavior:

1. python run_squad_w_distillation.py --model_type roberta --model_name_or_path distilroberta-base --output_dir ./distil_roberta --teacher_type roberta --teacher_name_or_path $ROBERTA_MODEL --train_file $SQUAD_DIR/train-v2.0.json --predict_file $SQUAD_DIR/dev-v2.0.json --version_2_with_negative --do_train --do_eval --do_lower_case --save_steps 5000 --logging_steps 5000

```python
/opt/conda/conda-bld/pytorch_1573049304260/work/aten/src/THC/THCTensorIndex.cu:361: void indexSelectLargeIndex(TensorInfo<T, IndexType>, TensorInfo<T, IndexType>, TensorInfo<long, IndexType>, int, int, IndexType, IndexType, long) [with T = float, IndexType = unsigned int, DstDim = 2, SrcDim = 2, IdxDim = -2, IndexIsMajor = true]: block: [206,0,0], thread: [95,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
Traceback (most recent call last):
  File "run_squad_w_distillation.py", line 871, in <module>
    main()
  File "run_squad_w_distillation.py", line 813, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer, teacher=teacher)
  File "run_squad_w_distillation.py", line 207, in train
    "input_ids": batch[0],
  File "/media/data2/anaconda/envs/distill2/lib/python3.6/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/media/data2/anaconda/envs/distill2/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: transform: failed to synchronize: cudaErrorAssert: device-side assert triggered
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

To train properly and evaluate on predict file dataset with f1_score near roberta model f1_score(81.5)

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.3.0
- Platform: "CentOS Linux 7"
- Python version: 3.6.9
- PyTorch version (GPU?): Yes
- Tensorflow version (GPU?): No
- Using GPU in script?: Yes, CUDA Version: 10.2, Driver Version: 440.33.01
- Using distributed or parallel set-up in script?: No


Update modeling_tf_utils.py

Tensorflow does not use .eval() vs .train().

closes https://github.com/huggingface/transformers/issues/2906


Loading tensorflow first and then loading transformers errors

TensorFlow

Anything TensorFlow

# 🐛 Bug

## Information

Model I am using: Bert

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

Run:
```
import tensorflow as tf
from transformers import TFBertForSequenceClassification
model = TFBertForSequenceClassification.from_pretrained('/path/to/my/tf/model/', from_pt = True)
```
Will produce the following output (with error):
```
>>> import tensorflow as tf
2020-02-20 09:36:51.035083: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-02-20 09:36:51.036337: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
>>> from transformers import TFBertForSequenceClassification
>>> model = TFBertForSequenceClassification.from_pretrained('/path/to/my/tf/model/', from_pt = True)
2020-02-20 09:36:52.226797: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-02-20 09:36:52.230595: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one
 NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.231392: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:02:02.0 name: GRID RTX6000-24Q computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 23.88GiB deviceMemoryBandwidth: 625.94GiB/s
2020-02-20 09:36:52.231447: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-20 09:36:52.231475: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-20 09:36:52.233199: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-02-20 09:36:52.233465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-02-20 09:36:52.234866: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-02-20 09:36:52.235660: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-02-20 09:36:52.235707: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-02-20 09:36:52.235845: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one
 NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.236261: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.236765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-02-20 09:36:52.237022: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-02-20 09:36:52.241987: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3192500000 Hz
2020-02-20 09:36:52.242277: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xeb8bae0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-02-20 09:36:52.242294: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-02-20 09:36:52.435669: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.436129: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0xec01900 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-02-20 09:36:52.436153: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GRID RTX6000-24Q, Compute Capability 7.5
2020-02-20 09:36:52.436350: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.436672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:02:02.0 name: GRID RTX6000-24Q computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 23.88GiB deviceMemoryBandwidth: 625.94GiB/s
2020-02-20 09:36:52.436706: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-20 09:36:52.436716: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-20 09:36:52.436744: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-02-20 09:36:52.436755: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-02-20 09:36:52.436765: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-02-20 09:36:52.436774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-02-20 09:36:52.436781: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-02-20 09:36:52.436861: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.437204: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.437493: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-02-20 09:36:52.437528: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-20 09:36:52.936429: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-02-20 09:36:52.936466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-02-20 09:36:52.936474: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-02-20 09:36:52.936737: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.937283: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:36:52.937654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21423 MB memory) -> physical GPU (device: 0, name: GRID RTX6000-24Q, pci bus id: 0000:02:02.0, compute capability: 7.5)
2020-02-20 09:36:54.066446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-20 09:36:54.066688: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2020-02-20 09:36:54.066725: E tensorflow/stream_executor/cuda/cuda_blas.cc:238] failed to create cublas handle: CUBLAS_STATUS_NOT_INITIALIZED
2020-02-20 09:36:54.066732: W tensorflow/stream_executor/stream.cc:2041] attempting to perform BLAS operation using StreamExecutor without BLAS support
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_utils.py", line 345, in from_pretrained
    return load_pytorch_checkpoint_in_tf2_model(model, resolved_archive_file, allow_missing_keys=True)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_pytorch_utils.py", line 93, in load_pytorch_checkpoint_in_tf2_model
    tf_model, pt_state_dict, tf_inputs=tf_inputs, allow_missing_keys=allow_missing_keys
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_pytorch_utils.py", line 125, in load_pytorch_weights_in_tf2_model
    tf_model(tf_inputs, training=False)  # Make sure model is built
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_bert.py", line 916, in call
    outputs = self.bert(inputs, **kwargs)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_bert.py", line 567, in call
    encoder_outputs = self.encoder([embedding_output, extended_attention_mask, head_mask], training=training)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_bert.py", line 376, in call
    layer_outputs = layer_module([hidden_states, attention_mask, head_mask[i]], training=training)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_bert.py", line 352, in call
    attention_outputs = self.attention([hidden_states, attention_mask, head_mask], training=training)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_bert.py", line 301, in call
    self_outputs = self.self_attention([input_tensor, attention_mask, head_mask], training=training)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/transformers/modeling_tf_bert.py", line 230, in call
    mixed_query_layer = self.query(hidden_states)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 822, in __call__
    outputs = self.call(cast_inputs, *args, **kwargs)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/keras/layers/core.py", line 1131, in call
    outputs = standard_ops.tensordot(inputs, self.kernel, [[rank - 1], [0]])
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py", line 4106, in tensordot
    ab_matmul = matmul(a_reshape, b_reshape)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/ops/math_ops.py", line 2798, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/ops/gen_math_ops.py", line 5616, in mat_mul
    _ops.raise_from_not_ok_status(e, name)
  File "/my_lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 6606, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InternalError: Blas GEMM launch failed : a.shape=(15, 768), b.shape=(768, 768), m=15, n=768, k=768 [Op:MatMul] name: tf_bert_for_sequence_classification/bert/encoder/layer_._0/attention/self/query/Tensordot/MatMul/
>>>
```

However, if I load transformers first and then load tensorflow, there is no problem...
(Output from console):
```
>>> from transformers import TFBertForSequenceClassification
2020-02-20 09:40:54.413603: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-02-20 09:40:54.414946: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
>>> import tensorflow as tf
>>> model = TFBertForSequenceClassification.from_pretrained('/path/to/my/tf/model/', from_pt = True)
2020-02-20 09:40:55.402943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-02-20 09:40:55.407404: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one
 NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.407771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:02:02.0 name: GRID RTX6000-24Q computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 23.88GiB deviceMemoryBandwidth: 625.94GiB/s
2020-02-20 09:40:55.407828: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-20 09:40:55.407858: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-20 09:40:55.409288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-02-20 09:40:55.409560: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-02-20 09:40:55.410954: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-02-20 09:40:55.411852: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-02-20 09:40:55.411906: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-02-20 09:40:55.412020: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one
 NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.412437: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.412712: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-02-20 09:40:55.412957: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-02-20 09:40:55.417720: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3192500000 Hz
2020-02-20 09:40:55.417908: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5be91f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-02-20 09:40:55.417927: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-02-20 09:40:55.604909: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.605396: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x5cc07b0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-02-20 09:40:55.605419: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GRID RTX6000-24Q, Compute Capability 7.5
2020-02-20 09:40:55.605632: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.605947: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:02:02.0 name: GRID RTX6000-24Q computeCapability: 7.5
coreClock: 1.77GHz coreCount: 72 deviceMemorySize: 23.88GiB deviceMemoryBandwidth: 625.94GiB/s
2020-02-20 09:40:55.605984: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-02-20 09:40:55.606000: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-02-20 09:40:55.606032: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-02-20 09:40:55.606045: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-02-20 09:40:55.606058: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-02-20 09:40:55.606070: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-02-20 09:40:55.606080: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-02-20 09:40:55.606159: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.606493: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:40:55.606763: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-02-20 09:41:00.803464: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-02-20 09:41:00.803503: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-02-20 09:41:00.803509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-02-20 09:41:00.803804: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:41:00.804291: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:981] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-02-20 09:41:00.804643: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 20754 MB memory) -> physical GPU (device: 0, name: GRID RTX6000-24Q, pci bus id: 0000:02:02.0, compute capability: 7.5)
>>>
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.0
- Platform: Linux
- Python version: 3.7.5
- PyTorch version (GPU?):
- Tensorflow version (GPU?): 2.1.0
- Using GPU in script?:Yes
- Using distributed or parallel set-up in script?: No


Tokenizer fast warnings

- Warning abut padding should not trigger so often now, especially when no padding strategy is provided by the user.

- RoberTa warning is now in RoberTaTokenizer, not GPT2 base class.


Expose all constructor parameters for BertTokenizerFast

Signed-off-by: Morgan Funtowicz <morgan@huggingface.co>


Error arises when using pipeline with community model

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using is: `mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es`

Language I am using the model on: Spanish

The problem arises when using:
* [ ] the official example scripts:
* [x] my own modified scripts:

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task:
* [x] my own task or dataset:

Steps to reproduce the behavior:

```python
from transformers import *

# Build a pipeline for QA
nlp = pipeline('question-answering', model='mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es',
               tokenizer='mrm8488/distill-bert-base-spanish-wwm-cased-finetuned-spa-squad2-es')

nlp(
    {
      'question': 'que queso es?',
      'context': 'Se utilizo en el dia de hoy un queso Emmental'
    }
)
```

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

This was working two days ago.

<details>
  <summary>Error log</summary>

  ```html
convert squad examples to features:   0%|          | 0/1 [00:00<?, ?it/s]WARNING:transformers.tokenization_utils:Disabled padding because no padding token set (pad_token: [PAD], pad_token_id: 1).
To remove this error, you can add a new pad token and then resize model embedding:
	tokenizer.pad_token = '<PAD>'
	model.resize_token_embeddings(len(tokenizer))
---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/usr/local/lib/python3.6/dist-packages/transformers/data/processors/squad.py", line 141, in squad_convert_example_to_features
    truncation_strategy="only_second" if tokenizer.padding_side == "right" else "only_first",
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 1796, in encode_plus
    **kwargs,
  File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 1722, in batch_encode_plus
    tokens = self._tokenizer.encode(*batch_text_or_text_pairs[0])
  File "/usr/local/lib/python3.6/dist-packages/tokenizers/implementations/base_tokenizer.py", line 141, in encode
    return self._tokenizer.encode(sequence, pair)
TypeError
"""

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
<ipython-input-21-df466dea770c> in <module>()
      8     nlp({
      9     'question': question,
---> 10     'context': context
     11     })
     12 )

11 frames
/usr/local/lib/python3.6/dist-packages/tokenizers/implementations/base_tokenizer.py in encode()
    139             An Encoding
    140         """
--> 141         return self._tokenizer.encode(sequence, pair)
    142
    143     def encode_batch(self, sequences: List[Union[str, Tuple[str, str]]]) -> List[Encoding]:

TypeError:
  ```
</details>

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.0
- Python version:3.6.9
- Torch version (GPU?): 1.4.0, running on CPU


Fast tokenizers ignore `add_special_tokens=False`

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
from transformers import AutoTokenizer

pretrained_model_name = "bert-base-cased"

fast_tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name)
slow_tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name, use_fast=False)

text = "hello"

assert fast_tokenizer.encode(text, add_special_tokens=False) == slow_tokenizer.encode(text, add_special_tokens=False)
```

## Expected behavior

The fast tokenizers shouldn't add the special tokens if `add_special_tokens` is equal to `False`.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.0
- Platform: Linux-4.15.0-76-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.4
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.0.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Fast Tokenizers save pretrained should return the list of generated file paths.


Breaking-change behavior in BERT tokenizer when stripping accents

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): Bert (could happen with other ones, don't know)

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

```python
from transformers import AutoTokenizer

pretrained_model_name = "bert-base-cased"

fast_tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name)
slow_tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name, use_fast=False)

text = "naïve"

assert fast_tokenizer.encode(text) == slow_tokenizer.encode(text)
```

With the slow, it only strips accents if lowercase is enabled (maybe a bug?):

https://github.com/huggingface/transformers/blob/e67676424191e5935362e5fe7e04b5c317d706a9/src/transformers/tokenization_bert.py#L346

With the fast one, it'd never strip accents:

https://github.com/huggingface/tokenizers/blob/python-v0.5.0/bindings/python/tokenizers/implementations/bert_wordpiece.py#L23

https://github.com/huggingface/transformers/blob/e67676424191e5935362e5fe7e04b5c317d706a9/src/transformers/tokenization_bert.py#L557-L565

I'd be cool to have that flag also, in both tokenizers.

Finally, this warning seems odd for the simple code from above:

```pycon
>>> assert fast_tokenizer.encode(text) == slow_tokenizer.encode(text)
Disabled padding because no padding token set (pad_token: [PAD], pad_token_id: 0).
To remove this error, you can add a new pad token and then resize model embedding:
	tokenizer.pad_token = '<PAD>'
	model.resize_token_embeddings(len(tokenizer))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError
```

Maybe here the `if pad_to_max_length` should be nesting the rest of the if?

https://github.com/huggingface/transformers/blob/e67676424191e5935362e5fe7e04b5c317d706a9/src/transformers/tokenization_utils.py#L80-L95

Didn't check in the other transformer models.

## Expected behavior

1. The 2 tokenizer outputs (slow and fast) should be the same.
2. The tokenizers should allow you to choose if to strip accents or not.
3. That warning shouldn't appear, IMHO.

## Environment info

- `transformers` version: 2.5.0
- Platform: Linux-4.15.0-76-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.4
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.0.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


How to train a LM with a custom Dataset?

Ex: LM (Pretraining)

Related to language modeling pre-training

# ❓ Questions & Help

I'm attempting to build a LM following the tutorial here (https://huggingface.co/blog/how-to-train).

Unfortunately, it is incomplete.  It shows how to create a custom `Dataset` but not how to execute `run_language_modeling.py` so that it is used.

**Any chance we can get the full script for training the LM, included how to specify our custom dataset?**


How to train with variable number of candidates for multiple choice selection?

Help wanted

Extra attention is needed, help appreciated

Usage

General questions about the library

# ❓ Questions

## Details
I am trying to train `GPT2DoubleHeadsModel` for the tasks of generation and multiple-choice selection. My dataset has examples with variable number of candidates - some examples have 10 candidates, some have 15, etc.

I wanted to be able to create a single `TensorDataset` object for my dataset and train the model using a `DataLoader` wrapped around this dataset. But clearly, since the number of candidates varies across examples, I am unable to do so.

What is an appropriate way (or best practice) to train `GPT2DoubleHeadsModel` with such a dataset?


Add syntax highlighting to the BibTeX in README


make RobertaForMaskedLM implementation identical to fairseq

closes https://github.com/huggingface/transformers/issues/1874

The implementation of RoBERTa in `transformers` differs from the original implementation in [fairseq](https://github.com/pytorch/fairseq/tree/master/fairseq/models/roberta), as results showed (cf. https://github.com/huggingface/transformers/issues/1874). I have documented my findings here https://github.com/huggingface/transformers/issues/1874#issuecomment-588359143 and made the corresponding changes accordingly in this PR.

Someone should check, however, that removing `get_output_embeddings()` does not have any adverse side-effects.


Override build_inputs_with_special_tokens for fast tokenizers


missing "para" attribute in ARC dataset for multiple choice question answering model

# 🐛 Bug

## Information

Model I am using Roberta.

Language I am using the model on (English)

The problem arises when using:
* [ ] the official example scripts: (give details below)
**https://github.com/huggingface/transformers/blob/master/examples/utils_multiple_choice.py**

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
Multiple choice question answering

## To reproduce

Steps to reproduce the behavior:

1. run_multiple_choice.py
with parameters as specified in the documentation replacing the task name to arc

In the data, there is no such parameter called "para"

contexts=[
                            options[0]["para"].replace("_", ""),
                            options[1]["para"].replace("_", ""),
                            options[2]["para"].replace("_", ""),
                            options[3]["para"].replace("_", ""),
                        ],


`PreTrainedTokenizerFast.build_inputs_with_special_tokens` doesn't add the special tokens

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BertTokenizer (but seems to apply to most)

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

```python
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('bert-base-cased')
tokenizer.build_inputs_with_special_tokens(["abc", "def"])
```

The output is `['abc', 'def']`.

## Expected behavior

The output should include the `[CLS]` and `[SEP]` tokens.

The problem is neither `PreTrainedTokenizerFast` nor its subclasses override `build_inputs_with_special_tokens`.

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 2.5.0
- Platform: Linux
- Python version: 3.7.6
- PyTorch version (GPU?): 1.4.0, w/o GPU
- Tensorflow version (GPU?): -
- Using GPU in script?: No.
- Using distributed or parallel set-up in script?: No.


Add slow generate tests for pretrained lm models

Move implementation of slow hardcoded generate models to this PR from. Checkout previous discussion in PR #2885


Model I am using Roberta

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Help needed with interpretation of the MLP class

Core: Modeling

Internals of the library; Models.

Hello,

I am having some trouble understanding the MLP function used in the Hugging Face GPT-2, which is found [here](https://github.com/huggingface/transformers/blob/73028c5df0c28ca179fbe565482a9c2143787f61/src/transformers/modeling_gpt2.py#L204).

Q1. For MLP, why are we setting the n_state to be equal to 3072, which is 4 * n_embd?
Q2. Below is the definition for the MLP class:
```python
class MLP(nn.Module):
    def __init__(self, n_state, config):  # in MLP: n_state=3072 (4 * n_embd)
        super().__init__()
        nx = config.n_embd
        self.c_fc = Conv1D(n_state, nx)
        self.c_proj = Conv1D(nx, n_state)
        self.act = gelu_new
        self.dropout = nn.Dropout(config.resid_pdrop)
```
in the MLP definition above, what exactly do the lines ``` Conv1D(n_state, nx)``` (the object ```self.c_fc```), and ``` Conv1D(nx, n_state)```   (the object ```self.c_proj```) do?

Thank you,


documentation for TF models mentions non-existent methods

Documentation of `TFPreTrainedModel.from_pretrained` method mentions the `.model()` and `.eval()` methods that are not defined for tensorflow models:

> The model is set in evaluation mode by default using ``model.eval()`` (Dropout modules are deactivated)
> To train the model, you should first set it back in training mode with ``model.train()``

https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_tf_utils.py#L195


squad_convert_example_to_features does not work with CamembertTokenizer

Core: Tokenization

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using : CamemBERT
Language I am using the model on : French

The problem arises when using:
* [*] my own modified scripts: (give details below)

The tasks I am working on is:
* [*] an official GLUE/SQUaD task: SQUaD

## To reproduce

Steps to reproduce the behavior:
1 - Copy paste this and run it
```python
from transformers import CamembertTokenizer, SquadExample, squad_convert_examples_to_features

tokenizer = CamembertTokenizer.from_pretrained('camembert-base')
example = SquadExample(
    'example_id',
    "Q",
    "C D E F G H",
    "D",
    2,
    "title"
)
features, _ = squad_convert_examples_to_features(
    examples=[
        example
    ],
    tokenizer=tokenizer,
    max_seq_length=30,
    doc_stride=128,
    max_query_length=128,
    is_training=True,
    return_dataset="pt",
    threads=1,
)

tokens = tokenizer.convert_ids_to_tokens(tokenizer.encode(example.question_text, example.context_text))
doc_token = example.doc_tokens

print({tokens[k]: doc_token[v] for k, v in features[0].token_to_orig_map.items()})
# Outputs
# {'</s>': 'C', '▁C': 'D', '▁D': 'E', '▁E': 'F', '▁F': 'G', '▁G': 'H'}
# Should be
# {'▁C': 'C', '▁D': 'D', '▁E': 'E', '▁F': 'F', '▁G': 'G', '▁H': 'H'}
```

## Expected behavior

The resulting features mapping is shifted by one when using the CamembertTokenizer.

This seems to be caused by a weird check in the method `squad_convert_example_to_features`.  (`if "roberta" in str(type(tokenizer))`) is evaluated to False when using the CamembertTokenizer (which is adapted from RobertaTokenizer)

When I patch the line `if "roberta" in str(type(tokenizer))` by `if "roberta" in str(type(tokenizer)) or "camembert" in str(type(tokenizer))`, I get the expected behavior.

I do not really know what would be the best way to handle this problem.

## Environment info

- `transformers` version: 2.4.1
- Platform: MacOS
- Python version: 3.7.6
- PyTorch version : 1.4
- Tensorflow version : None
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


Update to include example of LM

The model files have been updated in order to include the classification layers, based on https://github.com/huggingface/transformers/issues/2901, and now can be also used as a LM.


Convert BERT to RoBERTa

Core: Modeling

Internals of the library; Models.

Usage

General questions about the library

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

Given that RoBERTa outperformed BERT on several tasks, yet having a slight architecture modification, I want to know if it is possible to convert a BERT pretrained model to RoBERTa.

I am working with a BERT pretrained model on a domain specific corpora, given that I don't have the resources to train RoBERTa from scratch I want to know if I can convert the model to RoBERTa. If yes, how do I go about it?


Pre-trained BERT-LM missing LM Head - returns random token predictions

Core: Modeling

Internals of the library; Models.

# 🐛 Bug

## Information

I released Greek BERT, almost a week ago and so far I'm exploring its use by running some benchmarks in Greek datasets. Although Greek BERT works just fine for sequence tagging (`AutoModelForTokenClassification`) and text classification (`AutoModelForSequenceClassification`), there are issues when we try to to use it as Language Model (`AutoModelWithLMHead`) in order to predict masked tokens. The bug was originally reported in (https://github.com/nlpaueb/greek-bert/issues/1) by  @jzbjyb.

## To reproduce

- `transformers` version:
- Platform: Linux / Mac OS
- Python version: 3.7
- PyTorch version (GPU?): 1.0.1
- Tensorflow version (GPU?): 2.1
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No

The model has been trained using the official BERT release (https://github.com/google-research/bert) originally converted from Tensorflow checkpoints using the library script:

```
python transformers/convert_bert_original_tf_checkpoint_to_pytorch.py --tf_checkpoint_path /home/ichalkidis/greek_bert/variables --bert_config_file=/home/ichalkidis/greek_bert/config.json --pytorch_dump_path=/home/ichalkidis/greek_bert/pytorch_model.bin
```

and then exported accompanied by the tokenizer files using:

```python
from transformers import BertModel
from transformers import BertConfig, BertTokenizer
model_path = '/home/ichalkidis/greek_bert/'

bert = BertModel.from_pretrained(model_path + 'pytorch_model.bin',
                                         config=BertConfig().from_pretrained(model_path + 'config.json'))
bert.save_pretrained('/home/ichalkidis/bert-base-greek-uncased-v1/')

tokenizer = BertTokenizer.from_pretrained(model_path+'vocab.txt')
tokenizer.save_pretrained('/home/ichalkidis/bert-base-greek-uncased-v1/')
```

You can replicate the inconsistent behaviour of the LM  with the following script:

```python
import torch
from transformers import *

text = 'Είναι ένας [MASK] άνθρωπος.'
tokenizer_greek = AutoTokenizer.from_pretrained('nlpaueb/bert-base-greek-uncased-v1')
lm_model_greek = AutoModelWithLMHead.from_pretrained('nlpaueb/bert-base-greek-uncased-v1')
input_ids = tokenizer_greek.encode(text)
print(tokenizer_greek.convert_ids_to_tokens(input_ids))
# ['[CLS]', 'ειναι', 'ενας', '[MASK]', 'ανθρωπος', '.', '[SEP]']
outputs = lm_model_greek(torch.tensor([input_ids]))[0]
print(tokenizer_greek.convert_ids_to_tokens(outputs[0, 3].max(0)[1].item()))
# the most plausible prediction for [MASK] is changing in every single run
```

It is obvious that the LM Head (layer) is missing from `pytorch_model.bin`, my main questions and report are:
* How we could preserve this layer moving from the TF checkpoint to the final PyTorch model?
* Is it possible to serve the model both in Pytorch and TF binaries?


pull from original


RobertaTokenizer different than fairseq for 'world'

Core: Tokenization

Internals of the library; Tokenization.

`pip install fairseq`
```
roberta = torch.hub.load('pytorch/fairseq', 'roberta.base')
rt = RobertaTokenizer.from_pretrained('roberta-base')
for ex in ['Hello world', ' Hello world', ' world', 'world', 'Hello', ' Hello']:
    print(f'{ex} fairseq: {roberta.encode(ex).tolist()}, Transformers: {rt.encode(ex,  add_prefix_space=True)}')

>>>

Hello world fairseq: [0, 31414, 232, 2], Transformers: [0, 20920, 232, 2]
 Hello world fairseq: [0, 20920, 232, 2], Transformers: [0, 20920, 232, 2]
 world fairseq: [0, 232, 2], Transformers: [0, 232, 2]
world fairseq: [0, 8331, 2], Transformers: [0, 232, 2]
Hello fairseq: [0, 31414, 2], Transformers: [0, 20920, 2]
 Hello fairseq: [0, 20920, 2], Transformers: [0, 20920, 2]
```
Notice that even the token "world" is different, but results always the same with leading spaces.
Is this related to @joeddav recent work?
h/t @pnpnpn for uncovering:)


Language modeling example script missing the next sentence predicion

The example script in `run_language_modeling.py` does not include the next sentence prediction for pre-training BERT. I was wondering if that is a) an oversight, b) for simplicity, or c) because you have found its impact to be non-significant?


save_pretrained doesn't work with GPT2FastTokenizer

Internals of the library; Tokenization.

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT2TokenizerFast

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [X ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```python
> from transformers import *
> tok = GPT2TokenizerFast.from_pretrained('distilgpt2')
> tok.save_pretrained('./')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/bilal/Documents/transformers/src/transformers/tokenization_utils.py", line 519, in save_pretrained
    vocab_files = self.save_vocabulary(save_directory)
  File "/Users/bilal/Documents/transformers/src/transformers/tokenization_utils.py", line 529, in save_vocabulary
    raise NotImplementedError
NotImplementedError
```

## Expected behavior

The tokenizer should be able to be saved

## Environment info

- `transformers` version: 2.4.1
- Platform: Darwin-19.3.0-x86_64-i386-64bit
- Python version: 3.7.5
- PyTorch version (GPU?): 1.3.1 (False)
- Tensorflow version (GPU?): 2.0.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


BertModel' object missing 'save_pretrained' attribute

I was attempting to download a pre-trained BERT model & save it to my cloud directory using Google Colab.

model.save_pretrained() seems to be missing completely for some reason.

Link to Colab notebook: https://colab.research.google.com/drive/1ix_nNhsd89nLfTy6Nyh-Ak8PHn1SYm-0

Here's my code:

```
!pip install pytorch_pretrained_bert

import torch
from pytorch_pretrained_bert import BertTokenizer, BertModel
import pandas as pd
import numpy as np

### Let's download a model and tokenizer
model = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

### Now let's save our model and tokenizer to a directory
model.save_pretrained('./models/')
tokenizer.save_pretrained('./models/')

```

Error:
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-18-1a3b2c8b8e82> in <module>()
     11
     12 ### Now let's save our model and tokenizer to a directory
---> 13 model.save_pretrained('./models/')
     14 tokenizer.save_pretrained('./models/')
     15

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
    574                 return modules[name]
    575         raise AttributeError("'{}' object has no attribute '{}'".format(
--> 576             type(self).__name__, name))
    577
    578     def __setattr__(self, name, value):
```

AttributeError: 'BertModel' object has no attribute 'save_pretrained'


Enable 'from transformers import AlbertMLMHead'

Discussed at https://github.com/huggingface/transformers/issues/2894

I'm writing a custom pretraining script that incorporates both the masked language modeling (MLM) and sentence order prediction (SOP) objectives. I'm able to use the TFAlbertForMaskedLM model for the MLM objective, but need access to the last_hidden_state to write my SOP objective. I can do this if I have a raw TFAlbertModel and write my own MLM objective, but would prefer to just create my own model from pre-existing modularized components.

I know that there's a lot of care taken in API design, and the team may have explicitly decided against this. But if it is an option, it would make the transformers repo much more extensible for research.


Allow import of model components, e.g. `from transformers import TFAlbertMLMHead`

# 🚀 Feature request

Expose model components such as custom layers like `TFAlbertMLMHead`, that are created in `modeling_tf_albert.py` and others.

## Motivation

I'm writing a custom pretraining script that incorporates both the masked language modeling (MLM) and sentence order prediction (SOP) objectives. I'm able to use the `TFAlbertForMaskedLM` model for the MLM objective, but need access to the `last_hidden_state` to write my SOP objective. I can do this if I have a raw `TFAlbertModel` and write my own MLM objective, but would prefer to just create my own model from pre-existing modularized components.

## Your contribution

I can contribute this, it would just be modifying `__init__.py`. I know that there's a lot of care taken in API design, and the team may have explicitly decided against this. But if it is an option, it would make the transformers repo much more extensible for research.


Pipeline Loading Models and Tokenizers

Core: Pipeline

Internals of the library; Pipeline.

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag:
     https://stackoverflow.com/questions/tagged/huggingface-transformers

     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Hi I'm trying to use 'fmikaelian/flaubert-base-uncased-squad' for question answering. I understand that I should load the model and the tokenizers. I'm not sure how should I do this.

My code is basically far

`
from transformers import pipeline, BertTokenizer

nlp = pipeline('question-answering', \
model='fmikaelian/flaubert-base-uncased-squad', \
tokenizer='fmikaelian/flaubert-base-uncased-squad')`

Most probably this can be solve with a two liner.

Many thanks

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:
https://stackoverflow.com/questions/60287465/pipeline-loading-models-and-tokenizers


Create README.md

model card

Related to pretrained model cards


Fix InputExample docstring


Support for torch-lightning in NER examples

Update of https://github.com/huggingface/transformers/pull/2816

This PR creates a new example coding style for the pytorch code.

* Uses pytorch-lightning for the underlying training.
* Separates out the base transformer loading from the individual training.
* Moves each individual example to its own directory.
* Move the code in the readme to bash scripts.
The only two new files are run_pl_ner.py and transformers_base.py.

The goal is to keep the same format as the original command-line. Most of the argument names are preserved. I have verified that for NER the results of the same on GPU.

There are several nice benefits of lightning -> somewhat nicer logging and library integration (e.g. wandb), auto-checkpointing. Mostly the goal though is code readability with identical functionality.

Tests I ran:

* make sure that the test results are identical.
* print test results after training.
* test multi-gpu and apex (multigpu gives a nice speedup)


Getting: AttributeError: 'BertTokenizer' object has no attribute 'encode'

Core: Tokenization

Internals of the library; Tokenization.

PyTorch

Anything PyTorch

# 🐛 Bug

## AttributeError: 'BertTokenizer' object has no attribute 'encode'

Model, I am using Bert

The language I am using the model on English

The problem arises when using:
```
    input_ids = torch.tensor([tokenizer.encode("raw_text", add_special_tokens=True)])
```

The tasks I am working on is:
```
##Text Summary for the following paragraph of text
 <code>
"['26The Indian organic market\nhave begun to disrupt the market with their one-of-a-kind \nofferings.', 'In an effort to promote a healthier lifestyle, these \n\nplayers are playing a pivotal role by providing consumers with \n\nwholesome organic produce.', 'Since the organic food segment is still at a nascent stage \nin India, both the Government and private players need \n\n\n\ninvolved.', 'The organic farming industry in India holds immense \n\npotential to grow, provided it receives steady investment \n\n\n\nlike incentivizing organic cultivation, food processing, \n\n\n\nof the challenges faced by the organic sector today can be \n\ngrouped into three heads:\n\nŁ \n\nlengthy procedures, international validity, inadequate \ncertifying agencies and inadequate supporting infrastructure \n\n\n\n\ncost of internal audits and documentation is approximately \n\n\n\nreduced, it is expensive for many small groups of farmers or \nindividual farmers.', 'Ł \nThere is also a gap in the \n\nrequirements.', 'Additionally, key trading partners have \ntraditionally demonstrated a lack of willingness to sign \n\nequivalence arrangements.', 'Ł \nThe \n\n\nprocess of the farm or crop cannot be placed in the organic \n\n\nharvest is sold as conventional crops, thereby causing the \nfarmer to incur a loss.', 'Ł \ncommodities: \nDairy products have a different standard while \nmeat has a different standard .', 'The process of standardization \n\nof organic coconut will be different from that of the value-\n\nadded products of coconut.', 'Therefore, a company having \n\nand maintain multiple records as per the applicable standards.', 'Ł \n\nnumber of producers in the world yet they cultivate less than \n1% of the organic area.', 'The conventional production system is \nmore lucrative given the land fragmentation.', 'Ł Lack of incentives for farmers: \nThe transition from \n\nconventional to organic farming is accompanied by high \ninput costs and low yields in the initial years.', 'The cost of \ngoing completely organic is quite high, due to the high cost \n\nof organic manure.', 'The commercially available bio-manure \nproducts may not be completely organic, and therefore the \n\n\nThis is one of the many reasons why farmers are skeptical \nwhen it comes to shifting from conventional to organic \nfarming.', 'In such cases, the farmers choose to play it safe by \n\npracticing conventional methods of farming.', 'Ł Lack of standardized organic agriculture inputs and subsidy \non organic inputs:\n Farmers also face an acute shortage of \nquality standardized organic agriculture inputs, which are \noften much more expensive than conventional agricultural \n\ninputs.', 'There are no subsidies from the Government on \nagriculture inputs, especially biofertilizers and biopesticides, \nmaking the cost of cultivation for organic farming quite high.', 'Unless the farmers use their own farm grown manure in \nlarge quantities, they are unable to meet the expenses.', 'Lack \nof proper organic inputs often results in low yield making \n\norganic farming unsustainable for the farmers.', 'Ł Lack of organic cultivation research and extension: \nThe \n\ncurrent research and extension on organic farming are much \nlesser than that on conventional farming.', 'There is a lack of \n\n\nStrong government support for producing non-GMO high \nyielding varieties and niche crops for organic farming \nunder different agro-ecological zones across India require \n\ninvestment in organic research and extension.', 'The extension \nservices are very limited for organic, for example, the ATMA \nscheme focuses more on conventional farming.', 'There is no \n\ntimely advisory available for organic pest and disease control \n\nmeasures.', 'Processor-level challenges\nŁ Supply chain issues: \nMany farmers are apprehensive of \n\norganic farming since it involves high production costs.', 'The emphasis on collection, transportation and storage of \nfresh organic produce is very high.', 'Due to relatively low \n\nvolumes, the marketing and distribution chain of organic food \n\nvery high.', 'For example, organic produce cannot be stored in \n\ngovernment warehouses that practice chemical treatment of \nstorage areas.', 'High demand and low supply further create \n\n\nthese products have higher price markups than conventional \nproducts.', 'Additionally, many sellers mix the produce from \ndifferent geographical regions to help attain a competitive \n\nprice, thus compromising the geographical origin norm.', 'Ł Lack of a proper organic supply chain is felt more acutely in \n\nhilly, tribal and remote areas that have a high potential for \n\ninfrastructure.', 'Ł Global competitiveness:\n A major challenge India faces is \n\nthat of increasing its share in the global organic food export \nmarket, in lieu of global competitiveness.', 'There often exists a \ndichotomy between international quality and safety standards \n\nand Indian organic stands, which puts Indian produce at a \ndisadvantage.', 'Ł Lack of proper branding and packaging: \n\nof organic products require separate packing material that is \nnatural and requires distinctive branding that distinguishes \norganic from conventional products.', 'At present, there is \n\nan absence of regulations on labeling standards.', 'There is \n34\n\n10, 201835']"
```

## To reproduce

Steps to reproduce the behavior:

1. In the first Imported torch
```Python
import torch
from pytorch_pretrained_bert import BertTokenizer, BertModel, BertForMaskedLM
import logging
```
2. defined models :
```
MODELS = [(BertModel,       BertTokenizer,       'bert-base-uncased') ]
```
3.
```
# Let's encode some text in a sequence of hidden-states using each model:
for model_class, tokenizer_class, pretrained_weights in MODELS:
    # Load pretrained model/tokenizer
    tokenizer = tokenizer_class.from_pretrained(pretrained_weights)
    model = model_class.from_pretrained(pretrained_weights)
```
4. If I am trying to encode with following code
```
    # Encode text
   <code> input_ids = torch.tensor([tokenizer.encode("raw_text", add_special_tokens=True)])  # Add special tokens takes care of adding [CLS], [SEP], <s>... tokens in the right way for each model.
    with torch.no_grad():
        last_hidden_states = model(input_ids)[0]
```
I am getting following error
```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-10-190085fa3098> in <module>
      1 # Encode text
----> 2 input_ids = torch.tensor([tokenizer.encode("raw_text", add_special_tokens=True)])  # Add special tokens takes care of adding [CLS], [SEP], <s>... tokens in the right way for each model.
      3 with torch.no_grad():
      4     last_hidden_states = model(input_ids)[0]  # Models outputs are now tuples

AttributeError: 'BertTokenizer' object has no attribute 'encode'
```
## Expected behavior
Tokenization should get completed
## Environment info

- `transformers` version: '0.6.2'
- Platform: Windows 10
- Python version: 3.5
- PyTorch version (GPU?): 1.1.0 no gpu
- Tensorflow version (GPU?): Tensorflow 2.0
- Using GPU in script?:No
- Using distributed or parallel set-up in script?:No


[WIP] Adapt lm generate fn for seq 2 seq models

From looking at the soon-to-be-added Bart model, I though the language generation could be conceptually adapted as shown below to be able to produce language from seq-to-seq models (Bart & T5).

So far this is not tested at all and only adapted for the `_generate_no_beam_search()` function. Also it still has to be checked whether this is compatible with T5.

Would be happy about feedback @sshleifer, @thomwolf


Regarding attention size returned by the model

Core: Modeling

Internals of the library; Models.

TensorFlow

Anything TensorFlow

hi, while doing output_attentions - True in the huggingface model.it return attention of size:(no. of heads,seq length,seq length)
can we configure it to return only the attention of the last 2 heads of the model.
please let me know.