---
license: mit
datasets:
- squad
language:
- en
pipeline_tag: text-classification
widget: 
- text: "question: What number comes after five? answer: four"
- text: "question: Which person is associated with Kanye West? answer: a tree"
- text: "question: When is US independence day from aliens? answer: 7/4/1996"
---

# kgourgou/bert-base-uncased-QA-classification

An experiment into classifying whether a pair of (question, answer) is valid. This is not a very good model at this point, but eventually such a
model could help with RAG. For a stronger model, check this one by [vectara](https://huggingface.co/vectara/hallucination_evaluation_model).

Input must be formatted as 

```
question: {your query}? answer: {your possible answer}
```

The output probabilities are for 

1. class 0 = the answer string couldn't be an answer to the question and
2. class 1 = the answer string could be an answer to the question. 

"Could be" should be interpreted as a type match, e.g., if the question requires the answer to be a person or a number or a date.

Examples: 

- "question: What number comes after five? answer: four" → this should be class 1 as the answer is a number (even if it's not the right number).
- "question: Which person is associated with Kanye West? answer: a tree" → this should be class 0 as a tree is not a person.


## Base model details 

The base model is bert-base-uncased. For this experiment, I only use the "squad" dataset after preprocessing it to bring it to the required format.