--- language: - en --- # Model Card for `answer-finder-v1-S-en` This model is a question answering model developed by Sinequa. It produces two lists of logit scores corresponding to the start token and end token of an answer. Model name: `answer-finder-v1-S-en` ## Supported Languages The model was trained and tested in the following languages: - English ## Scores | Metric | Value | |:--------------------------------------------------------------|-------:| | F1 Score on SQuAD v2 with Hugging Face evaluation pipeline | 79.4 | | F1 Score on SQuAD v2 with Haystack evaluation pipeline | 79.5 | ## Inference Time | GPU | Quantization type | Batch size 1 | Batch size 32 | |:------------------------------------------|:------------------|---------------:|---------------:| | NVIDIA A10 | FP16 | 1 ms | 10 ms | | NVIDIA A10 | FP32 | 3 ms | 43 ms | | NVIDIA T4 | FP16 | 2 ms | 22 ms | | NVIDIA T4 | FP32 | 5 ms | 130 ms | | NVIDIA L4 | FP16 | 2 ms | 12 ms | | NVIDIA L4 | FP32 | 5 ms | 62 ms | **Note that the Answer Finder models are only used at query time.** ## Gpu Memory usage | Quantization type | Memory | |:-------------------------------------------------|-----------:| | FP16 | 300 MiB | | FP32 | 550 MiB | Note that GPU memory usage only includes how much GPU memory the actual model consumes on an NVIDIA T4 GPU with a batch size of 32. It does not include the fix amount of memory that is consumed by the ONNX Runtime upon initialization which can be around 0.5 to 1 GiB depending on the used GPU. ## Requirements - Minimal Sinequa version: 11.10.0 - Minimal Sinequa version for using FP16 models and GPUs with CUDA compute capability of 8.9+ (like NVIDIA L4): 11.11.0 - [Cuda compute capability](https://developer.nvidia.com/cuda-gpus): above 5.0 (above 6.0 for FP16 use) ## Model Details ### Overview - Number of parameters: 33 million - Base language model: [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) - Insensitive to casing and accents ### Training Data - [SQuAD v2](https://rajpurkar.github.io/SQuAD-explorer/)