Questions Answering Example
Author: schipiga
Pull request: no

Hello folks! Can you provide simple example how to use pytorch bert with pretrained model for questions answering?

Comments:
Hi!

You can check this file that implements question answering on the SQuAD dataset: https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/examples/run_squad.py

Thank you very much!

How can we use pre trained BertForQuestionAnswering model? I have looked into BertForNextSentencePrediction and output of model makes sense given the input vector, but unable to find any good example on BertForQuestionAnswering.

Have you tried looking at the official [documentation](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertforquestionanswering) that provides a simple example for each model?

Hi @LysandreJik, the official example was not clear to me. I understood the part of encoding. But I am looking for something like, I will give a question and a paragraph which would contain the answer, and I need the model to predict the answer span. But in the example they have done it with a single sentence, which is quite confusing!

Hey @LysandreJik, Sorry my bad, didn't look at run_squad.py, it has been changed a lot since I saw it first during which BERT was only released! It is so good to see everything being integrated at a single place! Thanks for the great work you guys! ❤️ 

@Arjunsankarlal Glad you could get what you were looking for!

@Arjunsankarlal @LysandreJik can you guys help me with the example. I got an error when I ran the example given in the [documentation](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertforquestionanswering) when encoding the sequence, that tokernizer doesn't have attribute "encode". So I updated the code as follows:

`from pytorch_pretrained_bert import BertTokenizer, BertForQuestionAnswering
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForQuestionAnswering.from_pretrained('bert-base-uncased')

tokenized_text = tokenizer.tokenize("Hello, my dog is cute")


indexed_tokens = tokenizer.convert_tokens_to_ids(tokenized_text)

input_ids = torch.tensor([indexed_tokens]) # Batch size 1
start_positions = torch.tensor([1])
end_positions = torch.tensor([3])
outputs = model(input_ids, start_positions=start_positions, end_positions=end_positions)
print(outputs)`

this is the output
tensor(1.7739, grad_fn=<DivBackward0>)

I believe it's a loss but I don't understand the example as in how does it answer the question. Also there isn't any start and end span. Can you please explain the example. Much appreciated.

Hi @adilmukhtar82 , could you give a look at the [`run_squad.py` example](https://github.com/huggingface/pytorch-transformers/blob/master/examples/run_squad.py), it shows how to use several models to do question answering.

You should probably update your repository version to `pytorch-transformers` too, most of the examples on our documentation won't work with `pytorch_pretrained_bert`.

@LysandreJik Thanks I have updated the repository and example is working fine. I am confused about the example mentioned in [documentation](https://huggingface.co/pytorch-transformers/model_doc/bert.html#bertforquestionanswering) ("hello, my dog is cute") as to how does it do with single sentence and not paragraph along with it. 

A bit late but here you go - 
```
from transformers import DistilBertTokenizer, DistilBertForQuestionAnswering
import torch

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased',return_token_type_ids = True)
model = DistilBertForQuestionAnswering.from_pretrained('distilbert-base-uncased-distilled-squad')

context = "The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world."
question = "What was President Donald Trump's prediction?"
encoding = tokenizer.encode_plus(question, context)

input_ids, attention_mask = encoding["input_ids"], encoding["attention_mask"]
start_scores, end_scores = model(torch.tensor([input_ids]), attention_mask=torch.tensor([attention_mask]))

ans_tokens = input_ids[torch.argmax(start_scores) : torch.argmax(end_scores)+1]
answer_tokens = tokenizer.convert_ids_to_tokens(ans_tokens , skip_special_tokens=True)

all_tokens = tokenizer.convert_ids_to_tokens(input_ids)

print ("\nAnswer Tokens: ")
print (answer_tokens)

answer_tokens_to_string = tokenizer.convert_tokens_to_string(answer_tokens)

print ("\nFinal Answer : ")
print (answer_tokens_to_string)

```

Output is : 
Answer Tokens:
['some', 'states', 'would', 're', '##open', 'this', 'month']

Final Answer :
some states would reopen this month


@ramsrigouthamg Hey, could you maybe also provide a tensorflow example?

Thanks @ramsrigouthamg !

@mariusjohan there are PyTorch and TensorFlow examples in the [usage](https://huggingface.co/transformers/usage.html#extractive-question-answering) section of the documentation.

@LysandreJik The link is now updated to https://github.com/huggingface/transformers/blob/master/examples/question-answering/run_squad.py

@ramsrigouthamg  @LysandreJik 
Examples mentioned by you is giving following error:

```
Traceback (most recent call last):
  File "run_predict.py", line 30, in <module>
    answer_start_scores
TypeError: argmax(): argument 'input' (position 1) must be Tensor, not str

```
After doing investigation, I found that the following code is returning strings instead of integer indices:

@ramsrigouthamg 
start_scores, end_scores = model(torch.tensor([input_ids]), attention_mask=torch.tensor([attention_mask]))

@LysandreJik 
answer_start_scores, answer_end_scores = model(**inputs)

Values returned:
answer_start_scores = 'answer_start_scores'
answer_end_scores  = 'answer_end_scores '
start_scores = 'start_scores'
end_scores  = 'end_scores'

I have fine tuned bert-en-base model on squad v1.1 and want to write prediction code. Can you please help?

@saurabhhssaurabh I found the solution.
You just need to change  `answer_start_scores, answer_end_scores = model(**inputs)` to either 
`answer_start_scores, answer_end_scores = model(**inputs).values()` or `answer_start_scores, answer_end_scores = model(**inputs, return_dicts=True)`

I got it from here: https://stackoverflow.com/questions/64901831/huggingface-transformer-model-returns-string-instead-of-logits

Thank you, @JacobLoe