slim-q-gen-tiny-ov

slim-q-gen-tiny-ov is a specialized function calling model that implements a generative 'question' (e.g., 'q-gen') function, which takes a context passage as an input, and then generates as an output a python dictionary consisting of one key:

{'question': ['What was the amount of revenue in the quarter?']}

The model has been designed to accept one of three different parameters to guide the type of question-answer created: 'question' (generates a standard question), 'boolean' (generates a 'yes-no' question), and 'multiple choice' (generates a multiple choice question).

This is an OpenVino int4 quantized version of slim-q-gen, providing a very fast, very small inference implementation, optimized for AI PCs using Intel GPU, CPU and NPU.

Model Description

• Developed by: llmware
• Model type: tinyllama
• Parameters: 1.1 billion
• Model Parent: llmware/slim-q-gen
• Language(s) (NLP): English
• License: Apache 2.0
• Uses: Question generation from a context passage
• RAG Benchmark Accuracy Score: NA
• Quantization: int4

Model Card Contact

llmware on github

llmware on hf

llmware website