model update
Browse files- .gitattributes +1 -0
- README.md +138 -0
- added_tokens.json +3 -0
- config.json +54 -0
- eval/metric.first.answer.paragraph_answer.question.lmqg_qg_koquad.default.json +1 -0
- eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json +1 -0
- eval/samples.test.hyp.paragraph_answer.question.lmqg_qg_koquad.default.txt +0 -0
- eval/samples.validation.hyp.paragraph_answer.question.lmqg_qg_koquad.default.txt +0 -0
- generation_config.json +7 -0
- pytorch_model.bin +3 -0
- special_tokens_map.json +8 -0
- spiece.model +3 -0
- tokenizer.json +3 -0
- tokenizer_config.json +12 -0
- trainer_config.json +1 -0
.gitattributes
CHANGED
@@ -32,3 +32,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
32 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
33 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
34 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
32 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
33 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
34 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
35 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
@@ -0,0 +1,138 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
|
2 |
+
---
|
3 |
+
license: cc-by-4.0
|
4 |
+
metrics:
|
5 |
+
- bleu4
|
6 |
+
- meteor
|
7 |
+
- rouge-l
|
8 |
+
- bertscore
|
9 |
+
- moverscore
|
10 |
+
language: ko
|
11 |
+
datasets:
|
12 |
+
- lmqg/qg_koquad
|
13 |
+
pipeline_tag: text2text-generation
|
14 |
+
tags:
|
15 |
+
- question generation
|
16 |
+
widget:
|
17 |
+
- text: "1990년 영화 《 <hl> 남부군 <hl> 》에서 단역으로 영화배우 첫 데뷔에 이어 같은 해 KBS 드라마 《지구인》에서 단역으로 출연하였고 이듬해 MBC 《여명의 눈동자》를 통해 단역으로 출연하였다."
|
18 |
+
example_title: "Question Generation Example 1"
|
19 |
+
- text: "백신이 없기때문에 예방책은 <hl> 살충제 <hl> 를 사용하면서 서식 장소(찻찬 받침, 배수로, 고인 물의 열린 저장소, 버려진 타이어 등)의 수를 줄임으로써 매개체를 통제할 수 있다."
|
20 |
+
example_title: "Question Generation Example 2"
|
21 |
+
- text: "<hl> 원테이크 촬영 <hl> 이기 때문에 한 사람이 실수를 하면 처음부터 다시 찍어야 하는 상황이 발생한다."
|
22 |
+
example_title: "Question Generation Example 3"
|
23 |
+
model-index:
|
24 |
+
- name: vocabtrimmer/mt5-base-trimmed-ko-15000-koquad-qg
|
25 |
+
results:
|
26 |
+
- task:
|
27 |
+
name: Text2text Generation
|
28 |
+
type: text2text-generation
|
29 |
+
dataset:
|
30 |
+
name: lmqg/qg_koquad
|
31 |
+
type: default
|
32 |
+
args: default
|
33 |
+
metrics:
|
34 |
+
- name: BLEU4 (Question Generation)
|
35 |
+
type: bleu4_question_generation
|
36 |
+
value: 11.7
|
37 |
+
- name: ROUGE-L (Question Generation)
|
38 |
+
type: rouge_l_question_generation
|
39 |
+
value: 27.43
|
40 |
+
- name: METEOR (Question Generation)
|
41 |
+
type: meteor_question_generation
|
42 |
+
value: 28.76
|
43 |
+
- name: BERTScore (Question Generation)
|
44 |
+
type: bertscore_question_generation
|
45 |
+
value: 83.92
|
46 |
+
- name: MoverScore (Question Generation)
|
47 |
+
type: moverscore_question_generation
|
48 |
+
value: 82.94
|
49 |
+
---
|
50 |
+
|
51 |
+
# Model Card of `vocabtrimmer/mt5-base-trimmed-ko-15000-koquad-qg`
|
52 |
+
This model is fine-tuned version of [vocabtrimmer/mt5-base-trimmed-ko-15000](https://huggingface.co/vocabtrimmer/mt5-base-trimmed-ko-15000) for question generation task on the [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) (dataset_name: default) via [`lmqg`](https://github.com/asahi417/lm-question-generation).
|
53 |
+
|
54 |
+
|
55 |
+
### Overview
|
56 |
+
- **Language model:** [vocabtrimmer/mt5-base-trimmed-ko-15000](https://huggingface.co/vocabtrimmer/mt5-base-trimmed-ko-15000)
|
57 |
+
- **Language:** ko
|
58 |
+
- **Training data:** [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) (default)
|
59 |
+
- **Online Demo:** [https://autoqg.net/](https://autoqg.net/)
|
60 |
+
- **Repository:** [https://github.com/asahi417/lm-question-generation](https://github.com/asahi417/lm-question-generation)
|
61 |
+
- **Paper:** [https://arxiv.org/abs/2210.03992](https://arxiv.org/abs/2210.03992)
|
62 |
+
|
63 |
+
### Usage
|
64 |
+
- With [`lmqg`](https://github.com/asahi417/lm-question-generation#lmqg-language-model-for-question-generation-)
|
65 |
+
```python
|
66 |
+
from lmqg import TransformersQG
|
67 |
+
|
68 |
+
# initialize model
|
69 |
+
model = TransformersQG(language="ko", model="vocabtrimmer/mt5-base-trimmed-ko-15000-koquad-qg")
|
70 |
+
|
71 |
+
# model prediction
|
72 |
+
questions = model.generate_q(list_context="1990년 영화 《 남부군 》에서 단역으로 영화배우 첫 데뷔에 이어 같은 해 KBS 드라마 《지구인》에서 단역으로 출연하였고 이듬해 MBC 《여명의 눈동자》를 통해 단역으로 출연하였다.", list_answer="남부군")
|
73 |
+
|
74 |
+
```
|
75 |
+
|
76 |
+
- With `transformers`
|
77 |
+
```python
|
78 |
+
from transformers import pipeline
|
79 |
+
|
80 |
+
pipe = pipeline("text2text-generation", "vocabtrimmer/mt5-base-trimmed-ko-15000-koquad-qg")
|
81 |
+
output = pipe("1990년 영화 《 <hl> 남부군 <hl> 》에서 단역으로 영화배우 첫 데뷔에 이어 같은 해 KBS 드라마 《지구인》에서 단역으로 출연하였고 이듬해 MBC 《여명의 눈동자》를 통해 단역으로 출연하였다.")
|
82 |
+
|
83 |
+
```
|
84 |
+
|
85 |
+
## Evaluation
|
86 |
+
|
87 |
+
|
88 |
+
- ***Metric (Question Generation)***: [raw metric file](https://huggingface.co/vocabtrimmer/mt5-base-trimmed-ko-15000-koquad-qg/raw/main/eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json)
|
89 |
+
|
90 |
+
| | Score | Type | Dataset |
|
91 |
+
|:-----------|--------:|:--------|:-----------------------------------------------------------------|
|
92 |
+
| BERTScore | 83.92 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
93 |
+
| Bleu_1 | 27.39 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
94 |
+
| Bleu_2 | 20.25 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
95 |
+
| Bleu_3 | 15.29 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
96 |
+
| Bleu_4 | 11.7 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
97 |
+
| METEOR | 28.76 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
98 |
+
| MoverScore | 82.94 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
99 |
+
| ROUGE_L | 27.43 | default | [lmqg/qg_koquad](https://huggingface.co/datasets/lmqg/qg_koquad) |
|
100 |
+
|
101 |
+
|
102 |
+
|
103 |
+
## Training hyperparameters
|
104 |
+
|
105 |
+
The following hyperparameters were used during fine-tuning:
|
106 |
+
- dataset_path: lmqg/qg_koquad
|
107 |
+
- dataset_name: default
|
108 |
+
- input_types: paragraph_answer
|
109 |
+
- output_types: question
|
110 |
+
- prefix_types: None
|
111 |
+
- model: vocabtrimmer/mt5-base-trimmed-ko-15000
|
112 |
+
- max_length: 512
|
113 |
+
- max_length_output: 32
|
114 |
+
- epoch: 15
|
115 |
+
- batch: 16
|
116 |
+
- lr: 0.0005
|
117 |
+
- fp16: False
|
118 |
+
- random_seed: 1
|
119 |
+
- gradient_accumulation_steps: 4
|
120 |
+
- label_smoothing: 0.15
|
121 |
+
|
122 |
+
The full configuration can be found at [fine-tuning config file](https://huggingface.co/vocabtrimmer/mt5-base-trimmed-ko-15000-koquad-qg/raw/main/trainer_config.json).
|
123 |
+
|
124 |
+
## Citation
|
125 |
+
```
|
126 |
+
@inproceedings{ushio-etal-2022-generative,
|
127 |
+
title = "{G}enerative {L}anguage {M}odels for {P}aragraph-{L}evel {Q}uestion {G}eneration",
|
128 |
+
author = "Ushio, Asahi and
|
129 |
+
Alva-Manchego, Fernando and
|
130 |
+
Camacho-Collados, Jose",
|
131 |
+
booktitle = "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing",
|
132 |
+
month = dec,
|
133 |
+
year = "2022",
|
134 |
+
address = "Abu Dhabi, U.A.E.",
|
135 |
+
publisher = "Association for Computational Linguistics",
|
136 |
+
}
|
137 |
+
|
138 |
+
```
|
added_tokens.json
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"<hl>": 15001
|
3 |
+
}
|
config.json
ADDED
@@ -0,0 +1,54 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"_name_or_path": "lmqg_output/trimmed_qg/mt5-base-trimmed-ko-15000-koquad-qg/model_dpyopu/epoch_5",
|
3 |
+
"add_prefix": false,
|
4 |
+
"architectures": [
|
5 |
+
"MT5ForConditionalGeneration"
|
6 |
+
],
|
7 |
+
"d_ff": 2048,
|
8 |
+
"d_kv": 64,
|
9 |
+
"d_model": 768,
|
10 |
+
"decoder_start_token_id": 0,
|
11 |
+
"dense_act_fn": "gelu_new",
|
12 |
+
"dropout_rate": 0.1,
|
13 |
+
"eos_token_id": 1,
|
14 |
+
"feed_forward_proj": "gated-gelu",
|
15 |
+
"initializer_factor": 1.0,
|
16 |
+
"is_encoder_decoder": true,
|
17 |
+
"is_gated_act": true,
|
18 |
+
"layer_norm_epsilon": 1e-06,
|
19 |
+
"model_type": "mt5",
|
20 |
+
"num_decoder_layers": 12,
|
21 |
+
"num_heads": 12,
|
22 |
+
"num_layers": 12,
|
23 |
+
"output_past": true,
|
24 |
+
"pad_token_id": 0,
|
25 |
+
"relative_attention_max_distance": 128,
|
26 |
+
"relative_attention_num_buckets": 32,
|
27 |
+
"tie_word_embeddings": false,
|
28 |
+
"tokenizer_class": "T5Tokenizer",
|
29 |
+
"torch_dtype": "float32",
|
30 |
+
"transformers_version": "4.26.1",
|
31 |
+
"use_cache": true,
|
32 |
+
"vocab_size": 15002,
|
33 |
+
"vocabtrimmer": {
|
34 |
+
"mining_config": {
|
35 |
+
"dataset": "mc4",
|
36 |
+
"dataset_column": "text",
|
37 |
+
"dataset_name": "ko",
|
38 |
+
"dataset_split": "validation",
|
39 |
+
"language": "ko",
|
40 |
+
"min_frequency": 2,
|
41 |
+
"target_vocab_size": 15000
|
42 |
+
},
|
43 |
+
"stats": {
|
44 |
+
"compression_rate_embedding": 5.9977130245649946,
|
45 |
+
"compression_rate_full": 37.992839576176756,
|
46 |
+
"parameter_size_embedding/raw": 384172032,
|
47 |
+
"parameter_size_embedding/trimmed": 23041536,
|
48 |
+
"parameter_size_full/raw": 582401280,
|
49 |
+
"parameter_size_full/trimmed": 221270784,
|
50 |
+
"vocab_size/raw": 250112,
|
51 |
+
"vocab_size/trimmed": 15001
|
52 |
+
}
|
53 |
+
}
|
54 |
+
}
|
eval/metric.first.answer.paragraph_answer.question.lmqg_qg_koquad.default.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"validation": {"Bleu_1": 0.2550327307052953, "Bleu_2": 0.18565919602820272, "Bleu_3": 0.13849787805956632, "Bleu_4": 0.10455582517645232}, "test": {"Bleu_1": 0.27074135776759123, "Bleu_2": 0.19984438513176458, "Bleu_3": 0.1508790452307565, "Bleu_4": 0.11543086461884584}}
|
eval/metric.first.sentence.paragraph_answer.question.lmqg_qg_koquad.default.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"validation": {"Bleu_1": 0.2874901609532262, "Bleu_2": 0.21295401906754355, "Bleu_3": 0.16086704096760465, "Bleu_4": 0.12262995630720323, "METEOR": 0.2906756591561873, "ROUGE_L": 0.27981872333332347, "BERTScore": 0.8312479596810421, "MoverScore": 0.8296273142630661}, "test": {"Bleu_1": 0.2739280981490131, "Bleu_2": 0.20249457516259292, "Bleu_3": 0.15293624325491237, "Bleu_4": 0.11698815032804398, "METEOR": 0.28759461092604915, "ROUGE_L": 0.27430471861095135, "BERTScore": 0.8392473354346395, "MoverScore": 0.8294259768954083}}
|
eval/samples.test.hyp.paragraph_answer.question.lmqg_qg_koquad.default.txt
ADDED
The diff for this file is too large to render.
See raw diff
|
|
eval/samples.validation.hyp.paragraph_answer.question.lmqg_qg_koquad.default.txt
ADDED
The diff for this file is too large to render.
See raw diff
|
|
generation_config.json
ADDED
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"_from_model_config": true,
|
3 |
+
"decoder_start_token_id": 0,
|
4 |
+
"eos_token_id": 1,
|
5 |
+
"pad_token_id": 0,
|
6 |
+
"transformers_version": "4.26.1"
|
7 |
+
}
|
pytorch_model.bin
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:7a23b3d9aa8ced5252ffa74542d6fdbeba7237177212274e19e0139d0f5711be
|
3 |
+
size 885186613
|
special_tokens_map.json
ADDED
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"additional_special_tokens": [
|
3 |
+
"<hl>"
|
4 |
+
],
|
5 |
+
"eos_token": "</s>",
|
6 |
+
"pad_token": "<pad>",
|
7 |
+
"unk_token": "<unk>"
|
8 |
+
}
|
spiece.model
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ef78f86560d809067d12bac6c09f19a462cb3af3f54d2b8acbba26e1433125d6
|
3 |
+
size 4309802
|
tokenizer.json
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:2d19ac9dd9ec96dd229b5d13eef16e28f3f5c930987123e2f770dd6f6140aa4d
|
3 |
+
size 1244356
|
tokenizer_config.json
ADDED
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
{
|
2 |
+
"additional_special_tokens": null,
|
3 |
+
"eos_token": "</s>",
|
4 |
+
"extra_ids": 0,
|
5 |
+
"model_max_length": 1000000000000000019884624838656,
|
6 |
+
"name_or_path": "lmqg_output/trimmed_qg/mt5-base-trimmed-ko-15000-koquad-qg/model_dpyopu/epoch_5",
|
7 |
+
"pad_token": "<pad>",
|
8 |
+
"sp_model_kwargs": {},
|
9 |
+
"special_tokens_map_file": "/home/patrick/.cache/torch/transformers/685ac0ca8568ec593a48b61b0a3c272beee9bc194a3c7241d15dcadb5f875e53.f76030f3ec1b96a8199b2593390c610e76ca8028ef3d24680000619ffb646276",
|
10 |
+
"tokenizer_class": "T5Tokenizer",
|
11 |
+
"unk_token": "<unk>"
|
12 |
+
}
|
trainer_config.json
ADDED
@@ -0,0 +1 @@
|
|
|
|
|
1 |
+
{"dataset_path": "lmqg/qg_koquad", "dataset_name": "default", "input_types": "paragraph_answer", "output_types": "question", "prefix_types": null, "model": "vocabtrimmer/mt5-base-trimmed-ko-15000", "max_length": 512, "max_length_output": 32, "epoch": 15, "batch": 16, "lr": 0.0005, "fp16": false, "random_seed": 1, "gradient_accumulation_steps": 4, "label_smoothing": 0.15}
|