slim-q-gen-tiny-ov

slim-q-gen-tiny-ov is a specialized function calling model that implements a generative 'question' (e.g., 'q-gen') function, which takes a context passage as an input, and then generates as an output a python dictionary consisting of one key:

{'question': ['What was the amount of revenue in the quarter?']}

The model has been designed to accept one of three different parameters to guide the type of question-answer created: 'question' (generates a standard question), 'boolean' (generates a 'yes-no' question), and 'multiple choice' (generates a multiple choice question).

This is an OpenVino int4 quantized version of slim-q-gen, providing a very fast, very small inference implementation, optimized for AI PCs using Intel GPU, CPU and NPU.

Model Description

  • Developed by: llmware
  • Model type: tinyllama
  • Parameters: 1.1 billion
  • Model Parent: llmware/slim-q-gen
  • Language(s) (NLP): English
  • License: Apache 2.0
  • Uses: Question generation from a context passage
  • RAG Benchmark Accuracy Score: NA
  • Quantization: int4

Model Card Contact

llmware on github

llmware on hf

llmware website

Downloads last month
20
Inference Examples
Inference API (serverless) has been turned off for this model.

Model tree for llmware/slim-q-gen-tiny-ov

Quantized
(1)
this model

Collection including llmware/slim-q-gen-tiny-ov