File size: 1,383 Bytes
83f8ffa 19a594a 83f8ffa |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
---
language: en
tags:
- t5
- text2text-generation
- openvino
- english
- query-expansion
- int8
pipeline_tag: text2text-generation
---
# Query Generation
The T5-base model was trained on the [MS MARCO Passage Dataset](https://github.com/microsoft/MSMARCO-Passage-Ranking), which consists of about 500k real search queries from Bing together with the relevant passage.
The model can be used for query expansion to learn semantic search models without requiring annotated training data: [Synthetic Query Generation](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/query_generation).
## Usage
```python
from optimum.intel import OVModelForSeq2SeqLM
from transformers import AutoTokenizer, pipeline
model_id = "SteveTran/T5-small-query-expansion-Q4"
model = OVModelForSeq2SeqLM.from_pretrained(model_id, use_cache=True, use_io_binding=False)
tokenizer = AutoTokenizer.from_pretrained(model_id)
instruction = "rewrite: "
prompt = "Who lived longer, Nikola Tesla or Milutin Milankovic?"
inputs = tokenizer(
["{} {}".format(instruction, prompt)],
padding=False,
return_tensors="pt",
)
outputs = model.generate(**inputs, max_new_tokens=24, use_cache=False, temperature=0.6, do_sample=True, top_p=0.95)
print("Answer: ", tokenizer.batch_decode(outputs, skip_special_tokens=True))
# Nikola Tesla vs Milutin Milankovic lifespan
``` |