Question and Answering With Bert | Towards Data Science

Asking a question to a machine and receiving an answer was always the stuff of sci-fi in the not too distant past. Now, things have changed, and we find ourselves using Q&A systems everywhere — without even realizing it.
Google search is the best example — although in most cases Google is used to find information and will simply point you in the right direction, many other queries are direct questions — to which Google often provides direct answers:
Of course, Google is ahead of the game. But that doesn’t mean that we can’t produce great Q&A systems — we can — and we can even use the exact same models that Google uses in their search.
In this article, we will learn how to use state-of-the-art transformer models for Q&A. We will take a look at Google’s Bert and how to use it.
In short, we will cover:
> HuggingFace's Transformers  - Installation> Setting-up a Q&A Transformer  - Finding a Model  - The Q&A Pipeline    1. Model and Tokenizer Initialization    2. Tokenization    3. Pipeline and Prediction
If you prefer video, I cover everything here too:
First things first — modern NLP is dominated by these incredible models called transformers. These models are brilliant, and a comparatively recent development (the first paper describing a transformer appeared in 2017).
Second, the real-world implementation of transformers is carried out almost exclusively using a library called transformers built by an incredible collection of people that refer to themselves as HuggingFace.
There are many reasons that the transformers library is so popular. But there are three factors, in particular, that come to mind:
Open-source — the library is open-source, and it has an incredibly active communityEase of use — we can begin using state-of-the-art models from Google, OpenAI, and Facebook in around five lines of codeModels — the library has an unimaginably huge number of models available that we can simply download and begin working with
Open-source — the library is open-source, and it has an incredibly active community
Ease of use — we can begin using state-of-the-art models from Google, OpenAI, and Facebook in around five lines of code
Models — the library has an unimaginably huge number of models available that we can simply download and begin working with
We can install transformers for Python easily with:
pip install transformers
However, the library relies on either PyTorch or TensorFlow being installed too — as these are used in the background.
So, we install TensorFlow with:
pip install tensorflowORconda install tensorflow
And PyTorch by following the instructions under ‘Install PyTorch’ provided here.
Now that we have transformers installed, we can get going with building our first Q&A transformer script.
Let’s first find a model to use. We head over to huggingface.co/models and click on Question-Answering to the left.
After that, we can find the two models we will be testing in this article — deepset/bert-base-cased-squad2 and deepset/electra-base-squad2.
Both of these models have been built by Deepset.AI — hence the deepset/. They have also both been pre-trained for Q&A on the SQuAD 2.0 dataset as denoted by squad2 at the end.
The first model is bert-base, meaning the base (not large, nor small) version of Bert. It is also cased meaning it considers uppercase/lowercase letters as distinct.
The second model is again the ‘base’ version of another model called Electra — which I have found is a genuinely amazing model for Q&A.
If you’re interested in swapping out Bert for Electra, just switch deepset/bert-base-cased-squad2 with deepset/electra-base-squad2 throughout this article. You can check out this article if you’re interested in learning more about how Electra works:
towardsdatascience.com
Okay great — all we need here is the model name deepset/bert-base-cased-squad2 — we will use this soon to load our model.
Our question-answering process at its core consists of three steps:
Model and tokenizer initializationQuery tokenizationPipeline and Prediction
Model and tokenizer initialization
Query tokenization
Pipeline and Prediction
These are the essentials — in reality, there are likely to be several other steps too, such as data pre-processing or context retrieval. But, in its simplest form, we only need to worry about these three steps.
Because we want to keep things as simple as possible for this introduction, we will only worry about these three steps.
1.Model and Tokenizer Initialization — this is our very first step, here we will import transformers and initialize our model and tokenizer using deepset/bert-base-cased-squad2.
This is the only code we need to load in our model and tokenizer — it’s incredibly simple.
2.Input Data Tokenization — We’ve initialized our tokenizer, and now it’s time to feed it some text to convert into Bert-readable token IDs.
Our Bert tokenizer is in-charge of converting human-readable text into Bert-friendly data, called token IDs.
First, the tokenizer takes a string and splits it into tokens. Which looks like:
"What organization is the IPCC a part of?"
["[CLS]", "What", "organization", "is", "the", "I", "P", "C", "C", "a", "part", "of", "?", "[SEP]", "[PAD]", "[PAD]"]
A few of these tokens are new, like [CLS] and [PAD] — these are special tokens, which are used to signify different things to Bert. See below for a full list of special tokens used by Bert.
After this initial tokenization, we convert this list of strings (tokens) into a list of integers (token IDs). This is done using an internal dictionary that contains every token understood by Bert.
Every one of these tokens maps to a unique integer value, a few of these mappings are listed in the special tokens table above. It is these token IDs that are fed into Bert.
3.Pipeline and Prediction — Now we’ve initialized our model and tokenizer, and understood how our tokenizer is converting our human-readable strings into lists of token IDs, we can move to integrating this and asking questions!
from transformers import pipelinenlp = pipeline('question-answering', model=model, tokenizer=tokenizer)
We can then begin asking questions with just:
Now, this is deceptively simple — but it is genuinely all we need to code a Q&A model and begin asking questions.
However, let’s break-down this pipeline function a little so we can understand what is actually happening.
First, we are passing the question and the context. When feeding these into a Q&A model, a specific format is expected — which looks like:
[CLS] <context> [SEP] <question> [SEP] [PAD] [PAD] ... [PAD]
So, in our example the merged input will look something like this:
Which will then be converted into a token ID format:
And it is this that will be passed along to Bert. At this point, Bert does some linguistic wizardry to identify the location of the answer within the context provided.
Once Bert has decided, it returns us the span of the answer — meaning the start token index, and end token index. Which in our case will look like this (up to “Answer span = [22:24]”):
Next up, our pipeline will pull out the relevant token IDs according to Bert’s start-end token index prediction. These token IDs will then be fed back into our tokenizer to be decoded into human-readable text!
And there we have it, our answer!
That’s it, we’ve built our first Q&A transformer model!
Of course, what we have done here is often just one part of a much larger puzzle. Most likely, there are a few more things you will need to consider, such as:
Context extraction — are you always going to have a one-two paragraph chunk of text containing your answer? Or is it more likely to be several pages or more? (FAISS + DPR are great for this)
Fine-tuning your model — the pre-trained models are great but can struggle when applied to Q&A where the language/formatting is slightly different (like financial terminology, or tweets).
These are some additional steps that will likely prove crucial when moving ahead and developing real-world use-cases — however, they’re not always required — and what we have built here represents the very core of the Q&A process.
If you’d like to learn more about Q&A transformers — like context extraction and fine-tuning — I have covered (and will be covering more on) these here on Medium — and on YouTube too.
Otherwise, I hope you’ve enjoyed the article! I’d love to hear your feedback, questions, or ideas — so feel free to leave a comment below or get in touch via Twitter.
Thanks for reading!
🤖 NLP With Transformers Course
*All images are by the author except where stated otherwise