asahi417 commited on
Commit
bc1b722
·
1 Parent(s): d25fa17

commit files to HF hub

Browse files
README.md CHANGED
@@ -31,33 +31,33 @@ model-index:
31
  metrics:
32
  - name: BLEU4 (Question Answering)
33
  type: bleu4_question_answering
34
- value: 28.92
35
  - name: ROUGE-L (Question Answering)
36
  type: rouge_l_question_answering
37
- value: 52.77
38
  - name: METEOR (Question Answering)
39
  type: meteor_question_answering
40
- value: 39.56
41
  - name: BERTScore (Question Answering)
42
  type: bertscore_question_answering
43
- value: 94.74
44
  - name: MoverScore (Question Answering)
45
  type: moverscore_question_answering
46
- value: 83.05
47
  - name: AnswerF1Score (Question Answering)
48
  type: answer_f1_score__question_answering
49
- value: 70.73
50
  - name: AnswerExactMatch (Question Answering)
51
  type: answer_exact_match_question_answering
52
- value: 49.15
53
  ---
54
 
55
  # Model Card of `vocabtrimmer/mt5-small-trimmed-ru-90000-ruquad-qa`
56
- This model is fine-tuned version of [vocabtrimmer/mt5-small-trimmed-ru-90000](https://huggingface.co/vocabtrimmer/mt5-small-trimmed-ru-90000) for question answering task on the [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
57
 
58
 
59
  ### Overview
60
- - **Language model:** [vocabtrimmer/mt5-small-trimmed-ru-90000](https://huggingface.co/vocabtrimmer/mt5-small-trimmed-ru-90000)
61
  - **Language:** ru
62
  - **Training data:** [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) (default)
63
  - **Online Demo:** [https://autoqg.net/](https://autoqg.net/)
@@ -93,16 +93,16 @@ output = pipe("question: чем соответствует абсолютная
93
 
94
  | | Score | Type | Dataset |
95
  |:-----------------|--------:|:--------|:-----------------------------------------------------------------|
96
- | AnswerExactMatch | 49.15 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
97
- | AnswerF1Score | 70.73 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
98
- | BERTScore | 94.74 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
99
- | Bleu_1 | 45.09 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
100
- | Bleu_2 | 39.13 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
101
- | Bleu_3 | 33.88 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
102
- | Bleu_4 | 28.92 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
103
- | METEOR | 39.56 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
104
- | MoverScore | 83.05 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
105
- | ROUGE_L | 52.77 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
106
 
107
 
108
 
@@ -114,10 +114,10 @@ The following hyperparameters were used during fine-tuning:
114
  - input_types: ['paragraph_question']
115
  - output_types: ['answer']
116
  - prefix_types: None
117
- - model: vocabtrimmer/mt5-small-trimmed-ru-90000
118
  - max_length: 512
119
  - max_length_output: 32
120
- - epoch: 15
121
  - batch: 32
122
  - lr: 0.001
123
  - fp16: False
 
31
  metrics:
32
  - name: BLEU4 (Question Answering)
33
  type: bleu4_question_answering
34
+ value: 32.59
35
  - name: ROUGE-L (Question Answering)
36
  type: rouge_l_question_answering
37
+ value: 56.49
38
  - name: METEOR (Question Answering)
39
  type: meteor_question_answering
40
+ value: 42.23
41
  - name: BERTScore (Question Answering)
42
  type: bertscore_question_answering
43
+ value: 95.43
44
  - name: MoverScore (Question Answering)
45
  type: moverscore_question_answering
46
+ value: 84.88
47
  - name: AnswerF1Score (Question Answering)
48
  type: answer_f1_score__question_answering
49
+ value: 75.0
50
  - name: AnswerExactMatch (Question Answering)
51
  type: answer_exact_match_question_answering
52
+ value: 53.0
53
  ---
54
 
55
  # Model Card of `vocabtrimmer/mt5-small-trimmed-ru-90000-ruquad-qa`
56
+ This model is fine-tuned version of [ckpts/mt5-small-trimmed-ru-90000](https://huggingface.co/ckpts/mt5-small-trimmed-ru-90000) for question answering task on the [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
57
 
58
 
59
  ### Overview
60
+ - **Language model:** [ckpts/mt5-small-trimmed-ru-90000](https://huggingface.co/ckpts/mt5-small-trimmed-ru-90000)
61
  - **Language:** ru
62
  - **Training data:** [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) (default)
63
  - **Online Demo:** [https://autoqg.net/](https://autoqg.net/)
 
93
 
94
  | | Score | Type | Dataset |
95
  |:-----------------|--------:|:--------|:-----------------------------------------------------------------|
96
+ | AnswerExactMatch | 53 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
97
+ | AnswerF1Score | 75 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
98
+ | BERTScore | 95.43 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
99
+ | Bleu_1 | 49.65 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
100
+ | Bleu_2 | 43.51 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
101
+ | Bleu_3 | 37.97 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
102
+ | Bleu_4 | 32.59 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
103
+ | METEOR | 42.23 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
104
+ | MoverScore | 84.88 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
105
+ | ROUGE_L | 56.49 | default | [lmqg/qg_ruquad](https://huggingface.co/datasets/lmqg/qg_ruquad) |
106
 
107
 
108
 
 
114
  - input_types: ['paragraph_question']
115
  - output_types: ['answer']
116
  - prefix_types: None
117
+ - model: ckpts/mt5-small-trimmed-ru-90000
118
  - max_length: 512
119
  - max_length_output: 32
120
+ - epoch: 17
121
  - batch: 32
122
  - lr: 0.001
123
  - fp16: False
eval/metric.first.answer.paragraph_question.answer.lmqg_qg_ruquad.default.json CHANGED
@@ -1 +1 @@
1
- {"validation": {"Bleu_1": 0.4665123059771689, "Bleu_2": 0.40847084954478935, "Bleu_3": 0.35703867230269826, "Bleu_4": 0.3072508578713082, "METEOR": 0.40165518849800647, "ROUGE_L": 0.5435572091700595, "BERTScore": 0.9506027232071442, "MoverScore": 0.8356314260080862, "AnswerF1Score": 72.43008302788581, "AnswerExactMatch": 50.377283558379666}, "test": {"Bleu_1": 0.4508723201110819, "Bleu_2": 0.3912739019011524, "Bleu_3": 0.3387826322351807, "Bleu_4": 0.28915608311983915, "METEOR": 0.39557574070630946, "ROUGE_L": 0.5277294386564438, "BERTScore": 0.9474412340484412, "MoverScore": 0.8305373857531934, "AnswerF1Score": 70.72807544532272, "AnswerExactMatch": 49.14614773629865}}
 
1
+ {"validation": {"Bleu_1": 0.5149703290774809, "Bleu_2": 0.45480765897104797, "Bleu_3": 0.4009811178398513, "Bleu_4": 0.34843674137594965, "METEOR": 0.4283961741388879, "ROUGE_L": 0.5775392506491396, "BERTScore": 0.956926358240857, "MoverScore": 0.8536981426014217, "AnswerF1Score": 76.49093650995337, "AnswerExactMatch": 54.20969023034154}, "test": {"Bleu_1": 0.4964820494316933, "Bleu_2": 0.43513081955408844, "Bleu_3": 0.3796910329312494, "Bleu_4": 0.32592761010234506, "METEOR": 0.42230936839877903, "ROUGE_L": 0.5648592283767015, "BERTScore": 0.9542533082797283, "MoverScore": 0.8487880679963307, "AnswerF1Score": 74.99535859704932, "AnswerExactMatch": 52.998411437648926}}
eval/samples.test.hyp.paragraph_question.answer.lmqg_qg_ruquad.default.txt CHANGED
The diff for this file is too large to render. See raw diff
 
eval/samples.validation.hyp.paragraph_question.answer.lmqg_qg_ruquad.default.txt CHANGED
The diff for this file is too large to render. See raw diff