Back to all models
Model card Files and versions Use in transformers
translation mask_token:
Query this model
πŸ”₯ This model is currently loaded and running on the Inference API. ⚠️ This model could not be loaded by the inference API. ⚠️ This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

⚑️ Upgrade your account to access the Inference API

Share Copied link to clipboard

Monthly model downloads

valhalla/distilt5-qg-hl-12-6 valhalla/distilt5-qg-hl-12-6
last 30 days



Contributed by

valhalla Suraj Patil
19 models

DistilT5 for question-generation

This is distilled version of t5-base-qg-hl model trained for answer aware question generation task. The answer spans are highlighted within the text with special highlight tokens.

The model is distilled using the No Teacher Distillation method proposed by Huggingface, here.

We just copy alternating layers from t5-base-qg-hl and finetune more on the same data. Following table lists other distilled models and their metrics.

distilt5-qg-hl-6-4 18.4141 24.8417 40.3435 - -
distilt5-qa-qg-hl-6-4 18.6493 24.9685 40.5605 76.13 84.659
distilt5-qg-hl-12-6 20.5275 26.5010 43.2676 - -
distilt5-qa-qg-hl-12-6 20.6109 26.4533 43.0895 81.61 89.831

You can play with the model using the inference API, just highlight the answer spans with <hl> tokens. For example

<hl> 42 <hl> is the answer to life, the universe and everything.

For more deatils see this repo.

Model in action πŸš€

You'll need to clone the repo.

Open In Colab

from pipelines import pipeline
nlp = pipeline("question-generation", model="valhalla/distilt5-qg-hl-12-6")
nlp("42 is the answer to life, universe and everything.")
=> [{'answer': '42', 'question': 'What is the answer to life, the universe and everything?'}]