ozcangundes commited on
Commit
22ebba6
1 Parent(s): cea9641

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +88 -0
README.md ADDED
@@ -0,0 +1,88 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: tr
3
+ datasets:
4
+ - TQUAD
5
+ pipeline_tag:
6
+ - question-answering
7
+ - question-generation
8
+ - multitask-model
9
+
10
+ license: apache-2.0
11
+ ---
12
+
13
+ # mT5-small based Turkish Multitask (Answer Extraction, Question Generation and Question Answering) System
14
+
15
+ [Google's Multilingual T5-small](https://github.com/google-research/multilingual-t5) is fine-tuned on [Turkish Question Answering dataset](https://github.com/okanvk/Turkish-Reading-Comprehension-Question-Answering-Dataset) for three downstream task **Answer extraction, Question Generation and Question Answering** served in this single model. mT5 model was also trained for multiple text2text NLP tasks.
16
+
17
+ All data processing, training and pipeline codes can be found on my [Github](https://github.com/ozcangundes/multitask-question-generation). I will share the training details in the repo as soon as possible.
18
+
19
+ mT5 small model has 300 million parameters and model size is about 1.2GB. Therefore, it takes significant amount of time to fine tune it. 8 epoch and 1e-4 learning rate with 0 warmup steps was applied during training. These hparams can be fine-tuned for much more better results.
20
+
21
+ ## Requirements ❗❗❗
22
+ ```
23
+ !pip install transformers==4.4.2
24
+ !pip install sentencepiece==0.1.95
25
+ !git clone https://github.com/ozcangundes/multitask-question-generation.git
26
+ %cd multitask-question-generation/
27
+ ```
28
+
29
+ ## Usage 🚀🚀
30
+ ```
31
+ from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
32
+ tokenizer = AutoTokenizer.from_pretrained("ozcangundes/mt5-multitask-qa-qg-turkish")
33
+ model = AutoModelForSeq2SeqLM.from_pretrained("ozcangundes/mt5-multitask-qa-qg-turkish")
34
+
35
+ from pipelines import pipeline #pipelines.py script in the cloned repo
36
+ multimodel = pipeline("multitask-qa-qg",tokenizer=tokenizer,model=model)
37
+
38
+ #sample text
39
+ text="Özcan Gündeş, 1993 yılı Tarsus doğumludur. Orta Doğu Teknik Üniversitesi \
40
+ Endüstri Mühendisliği bölümünde 2011 2016 yılları arasında lisans eğitimi görmüştür. \
41
+ Yüksek lisansını ise 2020 Aralık ayında, 4.00 genel not ortalaması ile \
42
+ Boğaziçi Üniversitesi, Yönetim Bilişim Sistemleri bölümünde tamamlamıştır.\
43
+ Futbolla yakından ilgilenmekle birlikte, Galatasaray kulübü taraftarıdır."
44
+ ```
45
+
46
+ ## Example - Both Question Generation and Question Answering 💬💬
47
+ ```
48
+ multimodel(text)
49
+
50
+ #output
51
+ => [{'answer': 'Tarsus', 'question': 'Özcan Gündeş nerede doğmuştur?'},
52
+ {'answer': '1993', 'question': 'Özcan Gündeş kaç yılında doğmuştur?'},
53
+ {'answer': '2011 2016',
54
+ 'question': 'Özcan Gündeş lisans eğitimini hangi yıllar arasında tamamlamıştır?'},
55
+ {'answer': 'Boğaziçi Üniversitesi, Yönetim Bilişim Sistemleri',
56
+ 'question': 'Özcan Gündeş yüksek lisansını hangi bölümde tamamlamıştır?'},
57
+ {'answer': 'Galatasaray kulübü',
58
+ 'question': 'Özcan Gündeş futbolla yakından ilgilenmekle birlikte hangi kulübü taraftarıdır?'}]
59
+ ```
60
+ From this text, 5 questions are generated and they are answered by the model.
61
+
62
+ ## Example - Question Answering 💭💭
63
+
64
+ Both text and also, related question should be passed into pipeline.
65
+ ```
66
+ multimodel({"context":text,"question":"Özcan hangi takımı tutmaktadır?"})
67
+
68
+ #output
69
+ => Galatasaray
70
+
71
+ multimodel({"context":text,"question":"Özcan, yüksek lisanstan ne zaman mezun oldu?"})
72
+
73
+ #output
74
+ => 2020 Aralık ayında
75
+
76
+
77
+ multimodel({"context":text,"question":"Özcan'ın yüksek lisans bitirme notu kaçtır?"})
78
+
79
+ #output
80
+ => 4.00
81
+
82
+ #Sorry for being cocky 😝😝
83
+ ```
84
+
85
+ ## ACKNOWLEDGEMENT
86
+
87
+ This work is inspired from [Suraj Patil's great repo](https://github.com/patil-suraj/question_generation). I would like to thank him for the clean codes and also,[Okan Çiftçi](https://github.com/okanvk) for the Turkish dataset 🙏
88
+