noamaanMulla-03/transformers-issues huggingface dataset:
How to speed up getting answers?
Author: MaheshChandrra
Pull request: no
Labels: name: wontfix

Hi ,

I'm facing the issues using BertForQuestionAnswering ,can you please help me fixing these:

1. I'm trying to use BertForQuestionAnswering pretrained model to get answers from news,
answer_question is the function where in you pass the context and the question and get the relevant answers,but if I have 100 contexts its taking around 100 seconds to get the answer,may I please know by any chance I can get the answers in much lesser time.

2. Some times I get the answer where in start index and end index are pointing to [SEP],that means the whole context,can I avoid this.


```
BERT_SQUAD = 'bert-large-uncased-whole-word-masking-finetuned-squad'

model = BertForQuestionAnswering.from_pretrained(BERT_SQUAD)
tokenizer = BertTokenizer.from_pretrained(BERT_SQUAD)

def answer_question(question, context):
    """
    Answer questions
    """
    try:
        print("Type:",type(context))
        print(context)

        encoded_dict = tokenizer.encode_plus(
                            question, context, # Sentence to encode.
                            add_special_tokens = True, # Add '[CLS]' and '[SEP]'
                            max_length = 256,  # Pad & truncate all sentences.
                            pad_to_max_length = True,
                            return_attention_mask = True,   # Construct attn. masks.
                            return_tensors = 'pt'     # Return pytorch tensors.
                       )
        print(encoded_dict)
        input_ids = encoded_dict['input_ids'].to(torch_device)
        token_type_ids = encoded_dict['token_type_ids'].to(torch_device) # segments

        start_scores, end_scores = model(input_ids, token_type_ids=token_type_ids)
        print('Start Scores:',start_scores)
        print('End Scores:',end_scores)
        all_tokens = tokenizer.convert_ids_to_tokens(input_ids[0])
        print(all_tokens)
        answer = tokenizer.convert_tokens_to_string(all_tokens[torch.argmax(start_scores) : torch.argmax(end_scores)+1])

        answer = answer.replace('[CLS]', '')
        #answer= answer.replace('[SEP]','')
        print("Start index:",all_tokens[torch.argmax(start_scores)])
        print("End index:",all_tokens[torch.argmax(end_scores)])
        print(answer)
    except ValueError:
        print("Error in fetching answer")
        answer=''
        
    
    return answer
```

Thanks in advance!!


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Implemented resizing of token embeddings for TensorFlow models
Author: RobMcH
Pull request: yes

As mentioned in [this issue](https://github.com/huggingface/transformers/issues/1838) transformers currently does not support resizing token embeddings with TensorFlow. I have implemented this functionality for ALBERT, BERT, DistilBERT, and GPT2.

**Note**
All of the respective TF[...]MainLayer[s] inherit from a utility class (TFLayerUtilsMixin; similar to the already existing TFModelUtilsMixin; both of them live in `modeling_tf_utils.py`) to avoid code duplication. This has to be done because TensorFlow models are structured differently than the corresponding PyTorch models - i.e., all the TF{ModelName} classes have a corresponding TF{ClassName}MainLayer which itself inherits from tf.keras.layers.Layer, whereas the PyTorch {ModelName} classes implement all the functionality themselves.

**Usage**
The usage is exactly the same as for the PyTorch models.
```
import transformers
bert = transformers.TFAutoModelForSequenceClassification.from_pretrained("bert-base-uncased")
tokenizer = transformers.AutoTokenizer.from_pretrained("bert-base-uncased")
tokenizer.add_tokens("[E1]"); tokenizer.add_tokens("[/E1]");
bert.resize_token_embeddings(len(tokenizer))
```

**Tests**
There are no tests in the existing transformers code for the `resize_token_embeddings` methods of the PyTorch models (as far as I can tell). This might be a separate issue.

Comments:
Hello !

Thanks a lot for this PR!! Can you rebase on master and push force in order to be able to review :)

Hi,

I've rebased on master. Are you able to review the PR like this?

Hello! Thanks a lot for your contribution, unfortunately we've decided to move on with https://github.com/huggingface/transformers/pull/4351 that was contributed earlier. 

Your contribution is still valuable, and we would have went with it had another PR not done it already. We look forward to your future PRs!
ImportError: cannot import name 'AutoModelForQuestionAnswering' from 'transformers 
Author: OguzKircicek
Pull request: no

Hi friends, 

I would like to use  transformers library. But while import I received this error. 

This code: 
```
from transformers import AutoTokenizer, AutoModelForQuestionAnswering, pipeline
import torch
 # LOAD MODEL
tokenizer = AutoTokenizer.from_pretrained("savasy/bert-base-turkish-squad")
model = AutoModelForQuestionAnswering.from_pretrained("savasy/bert-base-turkish-squad")

```
**Error**

**ImportError: cannot import name 'AutoModelForQuestionAnswering' from 'transformers' (C:\Users\oguzk\anaconda3\lib\site-packages\transformers\__init__.py)**

Comments:
What is your `transformers` version? Do you have PyTorch installed?

Thank you. I solved.

Great to hear!

> Thank you. I solved.

How did you solve it?

I installed pytorch library.
Typo in Reformer model card
Author: flozi00
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=h1) Report
> Merging [#5147](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f45e873910e60d89511ae0193711e71c5c710468&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5147/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5147      +/-   ##
==========================================
- Coverage   77.19%   77.19%   -0.01%     
==========================================
  Files         133      133              
  Lines       22233    22233              
==========================================
- Hits        17163    17162       -1     
- Misses       5070     5071       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5147/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `75.00% <0.00%> (-0.41%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=footer). Last update [f45e873...e655c44](https://codecov.io/gh/huggingface/transformers/pull/5147?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks a lot @flozi00 !
Fixing docs for Encoder Decoder Config
Author: mikaelsouza
Pull request: yes

This is a small contribution to the EncoderDecoderConfig documentation.
While trying to use this class, I noticed the documentation said that we could pass 2 "encoder" parameters to the class constructor. 
I am almost 99.99% certain it meant we could pass an encoder and a decoder parameter to the class constructor.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=h1) Report
> Merging [#5171](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/bc3a0c06075050d3de586c543e4ad6a7efc9260e&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5171/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5171   +/-   ##
=======================================
  Coverage   78.31%   78.31%           
=======================================
  Files         137      137           
  Lines       23475    23475           
=======================================
+ Hits        18384    18385    +1     
+ Misses       5091     5090    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/configuration\_encoder\_decoder.py](https://codecov.io/gh/huggingface/transformers/pull/5171/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2VuY29kZXJfZGVjb2Rlci5weQ==) | `100.00% <ø> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5171/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.16% <0.00%> (-0.13%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5171/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `86.11% <0.00%> (+0.29%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=footer). Last update [bc3a0c0...06ccbb9](https://codecov.io/gh/huggingface/transformers/pull/5171?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Oh, you are 100% correct here @mikaelsouza . This must be pretty confusing in the docs. Thanks for the fix!
fix bart doc
Author: fuzihaofzh
Pull request: yes

fix bart doc
change the model name in the example from "bart-large-cnn" to "facebook/bart-large-cnn" 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=h1) Report
> Merging [#5132](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/84be482f6698fac822a5113735f2242c6d3abc76&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5132/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5132   +/-   ##
=======================================
  Coverage   77.19%   77.19%           
=======================================
  Files         133      133           
  Lines       22232    22232           
=======================================
  Hits        17162    17162           
  Misses       5070     5070           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/5132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19iYXJ0LnB5) | `96.25% <ø> (ø)` | |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `85.81% <0.00%> (-0.15%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5132/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `75.40% <0.00%> (+0.40%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=footer). Last update [84be482...992cbe3](https://codecov.io/gh/huggingface/transformers/pull/5132?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Awesome thanks a lot :-)
How to find and use fine-tuned model for GLUE-CoLA? 
Author: 517030910405
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:

I am trying to use fine-tuned model for CoLA from GLUE. Is there a quick guide? Thanks

Comments:
Hi @517030910405 , you can search for the models on huggingface model hub here https://huggingface.co/models,
you can filter the models by task, CoLa is sequence classification task so you'll be able to find the model there.
Here's one I found which is trained on CoLA https://huggingface.co/textattack/bert-base-uncased-CoLA.

And to see how to use sequence classification models, check this usage guide https://huggingface.co/transformers/usage.html#sequence-classification

> Hi @517030910405 , you can search for the models on huggingface model hub here https://huggingface.co/models,
> you can filter the models by task, CoLa is sequence classification task so you'll be able to find the model there.
> Here's one I found which is trained on CoLA https://huggingface.co/textattack/bert-base-uncased-CoLA.
> 
> And to see how to use sequence classification models, check this usage guide https://huggingface.co/transformers/usage.html#sequence-classification

Thank you very much. The recommended fine-tuned model works very well.
Update glossary
Author: sgugger
Pull request: yes

Update the glossary to the new tokenizer API. I also added a first section about general terms that might confuse a beginner (which anyone is welcome to expand when they see a term that's not necessarily obvious and that we use a lot) and proper references to the subsections about the model inputs to make sure none of the links break.

There was a bad indent introduced since my cleanup of sphinx warnings, so fixed that in passing.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=h1) Report
> Merging [#5148](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f45e873910e60d89511ae0193711e71c5c710468&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5148/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5148   +/-   ##
=======================================
  Coverage   77.19%   77.20%           
=======================================
  Files         133      133           
  Lines       22233    22233           
=======================================
+ Hits        17163    17164    +1     
+ Misses       5070     5069    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/configuration\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/5148/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX2VsZWN0cmEucHk=) | `100.00% <ø> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5148/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.28% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=footer). Last update [f45e873...fde64f8](https://codecov.io/gh/huggingface/transformers/pull/5148?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


That's great! I left some nits.

Addressed (something wrong with the time in my WSL so the commit appears before the comments....)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
Author: ryonakamura
Pull request: no

When I convert a TensorFlow checkpoint in a `pytorch_model.bin` and run `run_classifier.py` with `--bert_model /path/to/pytorch_model.bin` option, following error occurs in `tokenization.py`.

```shell
12/10/2018 18:11:59 - INFO - pytorch_pretrained_bert.tokenization -   loading vocabulary file /Users/MAC/bert/model/uncased_L-12_H-768_A-12/pytorch_model.bin
Traceback (most recent call last):
  File "examples/run_classifier.py", line 637, in <module>
    main()
  File "examples/run_classifier.py", line 480, in main
    tokenizer = BertTokenizer.from_pretrained(args.bert_model, do_lower_case=args.do_lower_case)
  File "/Users/MAC/.pyenv/versions/anaconda3-5.3.0/lib/python3.6/site-packages/pytorch_pretrained_bert/tokenization.py", line 133, in from_pretrained
    tokenizer = cls(resolved_vocab_file, *inputs, **kwargs)
  File "/Users/MAC/.pyenv/versions/anaconda3-5.3.0/lib/python3.6/site-packages/pytorch_pretrained_bert/tokenization.py", line 76, in __init__
    self.vocab = load_vocab(vocab_file)
  File "/Users/MAC/.pyenv/versions/anaconda3-5.3.0/lib/python3.6/site-packages/pytorch_pretrained_bert/tokenization.py", line 51, in load_vocab
    token = reader.readline()
  File "/Users/MAC/.pyenv/versions/anaconda3-5.3.0/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
```

I noticed that `--bert_model` option is not path to `pytorch_model.bin` file but path to directory containing `pytorch_model.bin` and `vocab.txt`. I close it.


Comments:
I had the same question/confusion! Thanks for clarifying it should be the path to the directory and not the filename itself. 

Great help, thanks.

Thanks
[TFTrainer] Tensorflow Warning : experimental_run_v2 is deprecated
Author: astariul
Pull request: no

# 🐛 Bug

When running my code with TFTrainer, I'm receiving :

>WARNING - tensorflow -   From /home/me/.venv/bart/lib/python3.6/site-packages/transformers/trainer_tf.py:355: StrategyBase.experimental_run_v2 (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version.
Instructions for updating:
renamed to `run`

Is it expected ? How can I remove this warning ?

## Information

The problem arises when using:
* [ ] the official example scripts
* [x] my own modified scripts

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: CNN/DM
* [ ] my own task or dataset

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.11.0
- Platform: Linux-4.9.0-9-amd64-x86_64-with-debian-9.12
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.0 (False)
- Tensorflow version (GPU?): 2.2.0 (False)
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: Using TPU training


Comments:
closes by #4998
Accessing scores for the entire vocabulary in GPT2
Author: shampp
Pull request: no

I want to access the prediciton scores from the GPT2 model.

Follwoing the example given in the docstring, I am using the code

```
    output = model(input_ids,labels=input_ids)
    logit = output[1][0][-1,:].detach().numpy()
```
Is it correct ? I am expecting logit variable to be the same size as the vocabulary with corresponding scores.

Comments:
You would need to use `GPT2LMHeadModel` for that, and use a softmax layer to have scores. Here's an example:

```py
from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch

model = GPT2LMHeadModel.from_pretrained("gpt2")
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

inputs = tokenizer.encode("This is just", return_tensors="pt")
output = model(inputs)

model_output = output[0]
last_token_prediction = model_output[:, -1]
last_token_softmax = torch.softmax(last_token_prediction, dim=-1).squeeze()

n = 10
top_n_values = last_token_softmax.topk(n)

for index, value in zip(top_n_values.indices, top_n_values.values):
    print("Score: ", value.tolist())
    print("This is just" + tokenizer.decode(index.tolist()))
```
Roberta Tokenizer crashes when tokenizing  empty string in 2.8.0
Author: HaokunLiu
Pull request: no
Labels: name: wontfix

name: Core: Tokenization

name: Should Fix

```
import transformers

tokenizer = transformers.AutoTokenizer.from_pretrained("roberta-large")
tokenzer.tokenize("")
```

  File "/scratch/wh629/nlu/env/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 1054, in encode_plus
    first_ids = get_input_ids(text)
  File "/scratch/wh629/nlu/env/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 1028, in get_input_ids
    tokens = self.tokenize(text, add_special_tokens=add_special_tokens, **kwargs)
  File "/scratch/wh629/nlu/env/lib/python3.7/site-packages/transformers/tokenization_utils.py", line 759, in tokenize
    text = self.prepare_for_tokenization(text, **kwargs)
  File "/scratch/wh629/nlu/env/lib/python3.7/site-packages/transformers/tokenization_roberta.py", line 239, in prepare_for_tokenization
    if add_prefix_space and not text[0].isspace():
IndexError: string index out of range

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Fixed by #4209