Simple and fast Question Answering system using HuggingFace DistilBERT — single & batch inference examples provided. | by Ramsri Goutham | Towards Data Science

Question Answering systems have many use cases like automatically responding to a customer’s query by reading through the company’s documents and finding a perfect answer.
In this blog post, we will see how we can implement a state-of-the-art, super-fast, and lightweight question answering system using DistilBERT from Huggingface transformers library.
The input will be a short paragraph what we call context, and a question -
Context :The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world.Question:What was President Donald Trump's prediction?
The output from our question-answering system will be an answer from the context paragraph as shown below —
Answer: some states would reopen this month.
First, let’s see how we can answer a single question as shown above. Then we will see how we can leverage batch processing to answer multiple questions at once from the context.
Install the latest version of the transformers library -
pip install transformers==2.8.0pip install torch==1.4.0
Single Inference :
Here is the code to do single inference with DistilBERT:
The output will be :
Question  What was President Donald Trump's prediction?Answer Tokens:['some', 'states', 'would', 're', '##open', 'this', 'month']Answer :  some states would reopen this month
Now let’s try to do the same with batch inference where we try to pass three questions and get answers for them as a batch -
Three questions are -
1. What was President Donald Trump's prediction?2. How many deaths have been reported from the virus?3. How many cases have been reported in the United States?
Here is the code to do batch inference with DistilBERT :
The output will be —
Context :  The US has passed the peak on new coronavirus cases, President Donald Trump said and predicted that some states would reopen this month.The US has over 637,000 confirmed Covid-19 cases and over 30,826 deaths, the highest for any country in the world.Question:  What was President Donald Trump's prediction?Answer:  some states would reopen this monthQuestion:  How many deaths have been reported from the virus?Answer:  30 , 826Question:  How many cases have been reported in the United States?Answer:  over 637 , 000
If you are interested in automatic question generation from context rather than question answering, you can check out my various algorithms and open-sourced code in the links below -
True or False Question GenerationMultiple Choice Question GenerationGenerate pronoun questions for English language learningGrammar MCQ generation
True or False Question Generation
Multiple Choice Question Generation
Generate pronoun questions for English language learning
Grammar MCQ generation
Happy coding!
I launched a very interesting Udemy course titled “Question generation using NLP” expanding on some of the techniques discussed in this blog post. If you would like to take a look at it, here is the link.