model update

README.md

@@ -26,7 +26,7 @@ widget:
 - text: "extract answers: 지난 22일 아프리카TV는 BJ 철구가 서비스 정지 처분을 받았음을 밝혔다. 서비스 정지 처분을 사유는 철구가 10대 청소년에게 유해한 장면을 방송으로 내보냈기 때문이었다. 문제가 된 장면은 BJ 철구가 미성년자는 시청할 수 없게 하는 19세 시청 가능 설정을 하지 않은 채 흡연하는 모습을 여과 없이 드러낸 장면이다. 아프리카TV는 청소년 보호 정책의 '청소년들이 해로운 환경으로부터 보호받을 수 있도록 조치한다'라고 조항을 근거로 철구에게 서비스 정지 처분을 내렸다. 흡연 이외에 음주 방송 등도 19세 시청 가능 설정을 해야만 방송할 수 있다. <hl> 게다가 철구의 방송 정지 처분은 이번에 처음이 아니라 16번 째기 때문에 더욱더 논란이 되고 있다. <hl>"
   example_title: "Answer Extraction Example 2" 
 model-index:
-- name: lmqg/mt5-base-koquad-multitask
+- name: lmqg/mt5-base-koquad-qg-ae
   results:
   - task:
       name: Text2text Generation
@@ -51,34 +51,49 @@ model-index:
     - name: MoverScore (Question Generation)
       type: moverscore_question_generation
       value: 83.24
-    - name: QAAlignedF1Score-BERTScore
-      type: qa_aligned_f1_score_bertscore
+    - name: QAAlignedF1Score-BERTScore (Question & Answer Generation)
+      type: qa_aligned_f1_score_bertscore_question_answer_generation
       value: 80.28
-    - name: QAAlignedRecall-BERTScore
-      type: qa_aligned_recall_bertscore
+    - name: QAAlignedRecall-BERTScore (Question & Answer Generation)
+      type: qa_aligned_recall_bertscore_question_answer_generation
       value: 83.91
-    - name: QAAlignedPrecision-BERTScore
-      type: qa_aligned_precision_bertscore
+    - name: QAAlignedPrecision-BERTScore (Question & Answer Generation)
+      type: qa_aligned_precision_bertscore_question_answer_generation
       value: 77.03
-    - name: QAAlignedF1Score-MoverScore
-      type: qa_aligned_f1_score_moverscore
+    - name: QAAlignedF1Score-MoverScore (Question & Answer Generation)
+      type: qa_aligned_f1_score_moverscore_question_answer_generation
       value: 81.97
-    - name: QAAlignedRecall-MoverScore
-      type: qa_aligned_recall_moverscore
+    - name: QAAlignedRecall-MoverScore (Question & Answer Generation)
+      type: qa_aligned_recall_moverscore_question_answer_generation
       value: 86.43
-    - name: QAAlignedPrecision-MoverScore
-      type: qa_aligned_precision_moverscore
+    - name: QAAlignedPrecision-MoverScore (Question & Answer Generation)
+      type: qa_aligned_precision_moverscore_question_answer_generation
       value: 78.1
+    - name: BLEU4 (Answer Extraction)
+      type: bleu4_answer_extraction
+      value: 34.98
+    - name: ROUGE-L (Answer Extraction)
+      type: rouge_l_answer_extraction
+      value: 83.83
+    - name: METEOR (Answer Extraction)
+      type: meteor_answer_extraction
+      value: 61.26
+    - name: BERTScore (Answer Extraction)
+      type: bertscore_answer_extraction
+      value: 96.14
+    - name: MoverScore (Answer Extraction)
+      type: moverscore_answer_extraction
+      value: 95.2
     - name: AnswerF1Score (Answer Extraction)
-      type: answer_f1_score_answer_extraction
+      type: answer_f1_score__answer_extraction
       value: 88.43
     - name: AnswerExactMatch (Answer Extraction)
       type: answer_exact_match_answer_extraction
       value: 83.02
 ---
 
-# Model Card of `lmqg/mt5-base-koquad-multitask`
-This model is fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) for question generation task and answer extraction jointly on the [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
+# Model Card of `lmqg/mt5-base-koquad-qg-ae`
+This model is fine-tuned version of [google/mt5-base](https://huggingface.co/google/mt5-base) for question generation and answer extraction jointly on the [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
 
 
 ### Overview
@@ -95,7 +110,7 @@ This model is fine-tuned version of [google/mt5-base](https://huggingface.co/goo
 from lmqg import TransformersQG
 
 # initialize model
-model = TransformersQG(language="ko", model="lmqg/mt5-base-koquad-multitask")
+model = TransformersQG(language="ko", model="lmqg/mt5-base-koquad-qg-ae")
 
 # model prediction
 question_answer_pairs = model.generate_qa("1990년 영화 《 남부군 》에서 단역으로 영화배우 첫 데뷔에 이어 같은 해 KBS 드라마 《지구인》에서 단역으로 출연하였고 이듬해 MBC 《여명의 눈동자》를 통해 단역으로 출연하였다.")
@@ -106,7 +121,7 @@ question_answer_pairs = model.generate_qa("1990년 영화 《 남부군 》에
 ```python
 from transformers import pipeline
 
-pipe = pipeline("text2text-generation", "lmqg/mt5-base-koquad-multitask")
+pipe = pipeline("text2text-generation", "lmqg/mt5-base-koquad-qg-ae")
 
 # answer extraction
 answer = pipe("generate question: 1990년 영화 《 <hl> 남부군 <hl> 》에서 단역으로 영화배우 첫 데뷔에 이어 같은 해 KBS 드라마 《지구인》에서 단역으로 출연하였고 이듬해 MBC 《여명의 눈동자》를 통해 단역으로 출연하였다.")
@@ -119,7 +134,7 @@ question = pipe("extract answers: 또한 스피어스는 많은 새로운 여성
 ## Evaluation
 
 
-- ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/mt5-base-koquad-multitask/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json) 
+- ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/lmqg/mt5-base-koquad-qg-ae/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json) 
 
 |            |   Score | Type    | Dataset                                                          |
 |:-----------|--------:|:--------|:-----------------------------------------------------------------|
@@ -133,7 +148,7 @@ question = pipe("extract answers: 또한 스피어스는 많은 새로운 여성
 | ROUGE_L    |   28.55 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
 
 
-- ***Metric (Question & Answer Generation)***:  [raw metric file](https://huggingface.co/lmqg/mt5-base-koquad-multitask/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_koquad.default.json)
+- ***Metric (Question & Answer Generation)***:  [raw metric file](https://huggingface.co/lmqg/mt5-base-koquad-qg-ae/raw/main/eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_koquad.default.json)
 
 |                                 |   Score | Type    | Dataset                                                          |
 |:--------------------------------|--------:|:--------|:-----------------------------------------------------------------|
@@ -145,7 +160,7 @@ question = pipe("extract answers: 또한 스피어스는 많은 새로운 여성
 | QAAlignedRecall (MoverScore)    |   86.43 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
 
 
-- ***Metric (Answer Generation)***: [raw metric file](https://huggingface.co/lmqg/mt5-base-koquad-multitask/raw/main/eval/metric.first.answer.paragraph_sentence.answer.lmqg_qg_koquad.default.json)
+- ***Metric (Answer Extraction)***: [raw metric file](https://huggingface.co/lmqg/mt5-base-koquad-qg-ae/raw/main/eval/metric.first.answer.paragraph_sentence.answer.lmqg_qg_koquad.default.json)
 
 |                  |   Score | Type    | Dataset                                                          |
 |:-----------------|--------:|:--------|:-----------------------------------------------------------------|
@@ -181,7 +196,7 @@ The following hyperparameters were used during fine-tuning:
  - gradient_accumulation_steps: 2
  - label_smoothing: 0.15
 
-The full configuration can be found at [fine-tuning config file](https://huggingface.co/lmqg/mt5-base-koquad-multitask/raw/main/trainer_config.json).
+The full configuration can be found at [fine-tuning config file](https://huggingface.co/lmqg/mt5-base-koquad-qg-ae/raw/main/trainer_config.json).
 
 ## Citation
 ```

eval/metric.first.answer.paragraph.questions_answers.lmqg_qg_koquad.default.json

@@ -1,5 +1 @@
-<<<<<<< HEAD
-{"test": {"QAAlignedF1Score (BERTScore)": 0.8027605790024022, "QAAlignedRecall (BERTScore)": 0.8391231145490027, "QAAlignedPrecision (BERTScore)": 0.7702733584383046, "QAAlignedF1Score (MoverScore)": 0.8197272130915817, "QAAlignedRecall (MoverScore)": 0.8642533114826558, "QAAlignedPrecision (MoverScore)": 0.7810095393516585}, "validation": {"QAAlignedF1Score (BERTScore)": 0.8270067657961688, "QAAlignedRecall (BERTScore)": 0.8366250100173795, "QAAlignedPrecision (BERTScore)": 0.8181326643551915, "QAAlignedF1Score (MoverScore)": 0.8616471419521426, "QAAlignedRecall (MoverScore)": 0.8687903601401978, "QAAlignedPrecision (MoverScore)": 0.8559235660539652}}
-=======
 {"test": {"QAAlignedF1Score (BERTScore)": 0.8027605791423696, "QAAlignedRecall (BERTScore)": 0.8391231148330259, "QAAlignedPrecision (BERTScore)": 0.7702733584479435, "QAAlignedF1Score (MoverScore)": 0.8197379797060238, "QAAlignedRecall (MoverScore)": 0.8642724490753525, "QAAlignedPrecision (MoverScore)": 0.7810142692565831}, "validation": {"QAAlignedF1Score (BERTScore)": 0.8270067656226541, "QAAlignedRecall (BERTScore)": 0.8366250098715292, "QAAlignedPrecision (BERTScore)": 0.8181326641770484, "QAAlignedF1Score (MoverScore)": 0.8616624306894872, "QAAlignedRecall (MoverScore)": 0.8688060705285917, "QAAlignedPrecision (MoverScore)": 0.8559387631638197}}
->>>>>>> f6f982b5193f2a958e645f57b4457c79bdf5a15c

eval/metric.first.answer.paragraph_sentence.answer.lmqg_qg_koquad.default.json

@@ -1,5 +1 @@
-<<<<<<< HEAD
-{"validation": {"Bleu_1": 0.7234977041324892, "Bleu_2": 0.6232681100476516, "Bleu_3": 0.4884212653629802, "Bleu_4": 0.33498483094197157, "METEOR": 0.5963769689946691, "ROUGE_L": 0.8159680680273093, "BERTScore": 0.9524190997058138, "MoverScore": 0.9417069414119854, "AnswerF1Score": 84.91490896277583, "AnswerExactMatch": 79.48317724592438}, "test": {"Bleu_1": 0.7493322781678949, "Bleu_2": 0.6538723254802299, "Bleu_3": 0.5138896890145673, "Bleu_4": 0.3497542417446219, "METEOR": 0.6126328482686303, "ROUGE_L": 0.8383120221719403, "BERTScore": 0.9613852180822252, "MoverScore": 0.9520114109966031, "AnswerF1Score": 88.43194707215045, "AnswerExactMatch": 83.02115851543532}}
-=======
 {"validation": {"Bleu_1": 0.7234977041324892, "Bleu_2": 0.6232681100476516, "Bleu_3": 0.4884212653629802, "Bleu_4": 0.33498483094197157, "METEOR": 0.5963769689946691, "ROUGE_L": 0.8159680680273093, "BERTScore": 0.9524191018611327, "MoverScore": 0.9417059572948544, "AnswerF1Score": 84.91490896277583, "AnswerExactMatch": 79.48317724592438}, "test": {"Bleu_1": 0.7493322781678949, "Bleu_2": 0.6538723254802299, "Bleu_3": 0.5138896890145673, "Bleu_4": 0.3497542417446219, "METEOR": 0.6126328482686303, "ROUGE_L": 0.8383120221719403, "BERTScore": 0.9613852186352686, "MoverScore": 0.952047569907686, "AnswerF1Score": 88.43194707215045, "AnswerExactMatch": 83.02115851543532}}
->>>>>>> f6f982b5193f2a958e645f57b4457c79bdf5a15c