---
language: en
tags:
- t5
- text2text-generation
- openvino
- english
- query-expansion
- int8
pipeline_tag: text2text-generation
---
# Query Generation
The T5-base model was trained on the [MS MARCO Passage Dataset](https://github.com/microsoft/MSMARCO-Passage-Ranking), which consists of about 500k real search queries from Bing together with the relevant passage.

The model can be used for query expansion to learn semantic search models without requiring annotated training data: [Synthetic Query Generation](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/query_generation).


## Usage

```python
from optimum.intel import OVModelForSeq2SeqLM
from transformers import AutoTokenizer, pipeline

model_id = "SteveTran/T5-small-query-expansion-Q4"
model = OVModelForSeq2SeqLM.from_pretrained(model_id, use_cache=True, use_io_binding=False)
tokenizer = AutoTokenizer.from_pretrained(model_id)

instruction = "rewrite: "
prompt = "Who lived longer, Nikola Tesla or Milutin Milankovic?"
inputs = tokenizer(
    ["{} {}".format(instruction, prompt)],
    padding=False,
    return_tensors="pt",
)

outputs = model.generate(**inputs, max_new_tokens=24, use_cache=False, temperature=0.6, do_sample=True, top_p=0.95)
print("Answer: ", tokenizer.batch_decode(outputs, skip_special_tokens=True))
# Nikola Tesla vs Milutin Milankovic lifespan
```