Hugging Face's logo

lmqg
/
mt5-small-squad-qg

Text2Text Generation
Transformers
PyTorch
English
mt5
question generation
Eval Results
Inference Endpoints
Model card Files Community
1
asahi417 commited on
Commit
4616cbd
•
1 Parent(s): 37d6f58

model update

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -14,11 +14,11 @@ pipeline_tag: text2text-generation
14
  tags:
15
  - question generation
16
  widget:
17
- - text: "generate question: <hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records."
18
  example_title: "Question Generation Example 1"
19
- - text: "generate question: Beyonce further expanded her acting career, starring as blues singer <hl> Etta James <hl> in the 2008 musical biopic, Cadillac Records."
20
  example_title: "Question Generation Example 2"
21
- - text: "generate question: Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, <hl> Cadillac Records <hl> ."
22
  example_title: "Question Generation Example 3"
23
  model-index:
24
  - name: lmqg/mt5-small-squad
@@ -231,7 +231,7 @@ model_path = 'lmqg/mt5-small-squad'
231
  pipe = pipeline("text2text-generation", model_path)
232
 
233
  # Question Generation
234
- question = pipe('generate question: <hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.')
235
  ```
236
 
237
  ## Evaluation Metrics
@@ -241,7 +241,7 @@ question = pipe('generate question: <hl> Beyonce <hl> further expanded her actin
241
 
242
  | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
243
  |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
244
- | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) | default | 0.21650505934418166 | 0.489464328982525 | 0.23833897056449205 | 0.9000723844397448 | 0.62747065964027 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json) |
245
 
246
 
247
 
@@ -249,13 +249,13 @@ question = pipe('generate question: <hl> Beyonce <hl> further expanded her actin
249
 
250
  | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
251
  |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
252
- | [lmqg/qg_itquad](https://huggingface.co/datasets/lmqg/qg_itquad) | default | 0.005438910607183992 | 0.05010570221421983 | 0.05890828426558759 | 0.7260160158030385 | 0.5023119088393686 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_itquad.default.json) |
253
- | [lmqg/qg_jaquad](https://huggingface.co/datasets/lmqg/qg_jaquad) | default | 4.4114578660129224e-08 | 0.06084267343290677 | 0.005149267426183168 | 0.6608093198082075 | 0.46526108687696893 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_jaquad.default.json) |
254
- | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) | default | 4.229109829516021e-12 | 0.009881091250723615 | 0.017796529053904556 | 0.7089446693028568 | 0.49098728551715626 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_ruquad.default.json) |
255
- | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) | default | 9.242783121165897e-12 | 0.01556150764938016 | 0.04809700451843158 | 0.7353078946893743 | 0.5036973829954939 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_dequad.default.json) |
256
- | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) | default | 0.0059191752064594125 | 0.05208940592236566 | 0.06021086135293597 | 0.7494422899749911 | 0.5062373132800192 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_esquad.default.json) |
257
- | [lmqg/qg_frquad](https://huggingface.co/datasets/lmqg/qg_frquad) | default | 0.0171464639522496 | 0.1583673053928925 | 0.08244973027319356 | 0.7291012183458674 | 0.509610854598101 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_frquad.default.json) |
258
- | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) | default | 1.4750917137316939e-12 | 0.0006466767450454226 | 0.007310046912436679 | 0.6634288882769679 | 0.4586124640357038 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json) |
259
 
260
 
261
  ## Training hyperparameters
 
14
  tags:
15
  - question generation
16
  widget:
17
+ - text: "<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records."
18
  example_title: "Question Generation Example 1"
19
+ - text: "Beyonce further expanded her acting career, starring as blues singer <hl> Etta James <hl> in the 2008 musical biopic, Cadillac Records."
20
  example_title: "Question Generation Example 2"
21
+ - text: "Beyonce further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, <hl> Cadillac Records <hl> ."
22
  example_title: "Question Generation Example 3"
23
  model-index:
24
  - name: lmqg/mt5-small-squad
 
231
  pipe = pipeline("text2text-generation", model_path)
232
 
233
  # Question Generation
234
+ question = pipe('<hl> Beyonce <hl> further expanded her acting career, starring as blues singer Etta James in the 2008 musical biopic, Cadillac Records.')
235
  ```
236
 
237
  ## Evaluation Metrics
 
241
 
242
  | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
243
  |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
244
+ | [lmqg/qg_squad](https://huggingface.co/datasets/lmqg/qg_squad) | default | 0.217 | 0.489 | 0.238 | 0.9 | 0.627 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_squad.default.json) |
245
 
246
 
247
 
 
249
 
250
  | Dataset | Type | BLEU4 | ROUGE-L | METEOR | BERTScore | MoverScore | Link |
251
  |:--------|:-----|------:|--------:|-------:|----------:|-----------:|-----:|
252
+ | [lmqg/qg_itquad](https://huggingface.co/datasets/lmqg/qg_itquad) | default | 0.005 | 0.05 | 0.059 | 0.726 | 0.502 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_itquad.default.json) |
253
+ | [lmqg/qg_jaquad](https://huggingface.co/datasets/lmqg/qg_jaquad) | default | 0.0 | 0.061 | 0.005 | 0.661 | 0.465 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_jaquad.default.json) |
254
+ | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) | default | 0.0 | 0.01 | 0.018 | 0.709 | 0.491 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_ruquad.default.json) |
255
+ | [lmqg/qg_dequad](https://huggingface.co/datasets/lmqg/qg_dequad) | default | 0.0 | 0.016 | 0.048 | 0.735 | 0.504 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_dequad.default.json) |
256
+ | [lmqg/qg_esquad](https://huggingface.co/datasets/lmqg/qg_esquad) | default | 0.006 | 0.052 | 0.06 | 0.749 | 0.506 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_esquad.default.json) |
257
+ | [lmqg/qg_frquad](https://huggingface.co/datasets/lmqg/qg_frquad) | default | 0.017 | 0.158 | 0.082 | 0.729 | 0.51 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_frquad.default.json) |
258
+ | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) | default | 0.0 | 0.001 | 0.007 | 0.663 | 0.459 | [link](https://huggingface.co/lmqg/mt5-small-squad/raw/main/eval_ood/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json) |
259
 
260
 
261
  ## Training hyperparameters