Commit
•
293b74d
1
Parent(s):
8c10c0c
Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# DistilGPT2 Fine-Tuned on PubMedQA Artificial Subset
|
2 |
+
This model is a fine-tuned version of the DistilGPT2 for question-answering tasks in the biomedical domain. The model was trained on a subset of 50,000 artificial samples from the PubMedQA dataset.
|
3 |
+
|
4 |
+
## Model Details
|
5 |
+
1. Model architecture: DistilGPT2
|
6 |
+
2. Training dataset: 50,000 samples from PubMedQA artificial subset
|
7 |
+
3. Training epochs: 3
|
8 |
+
4. Tokenizer maximum length: 512
|
9 |
+
5. Fine-Tuning Details: Model finetuning was done for three epochs using a standard model training pipeline provided by the Huggingface library. During training, the tokenizer was configured with a maximum token length of 512.
|
10 |
+
|
11 |
+
# Example Usage
|
12 |
+
You can use this model for medical question-answering tasks by simply loading it with the Huggingface transformers library and providing a prompt.
|
13 |
+
```
|
14 |
+
from transformers import GPT2LMHeadModel, GPT2Tokenizer
|
15 |
+
|
16 |
+
tokenizer = GPT2Tokenizer.from_pretrained("DistilGPT_PubMedQA")
|
17 |
+
model = GPT2LMHeadModel.from_pretrained("DistilGPT_PubMedQA")
|
18 |
+
|
19 |
+
prompt = "question: What is the primary function of the liver? context: The liver is a vital organ that performs several essential functions, including detoxification, protein synthesis, and the production of biochemicals necessary for digestion."
|
20 |
+
|
21 |
+
input_ids = tokenizer.encode(prompt, return_tensors="pt")
|
22 |
+
output = model.generate(input_ids, max_length=100, num_return_sequences=1, no_repeat_ngram_size=2)
|
23 |
+
response = tokenizer.decode(output[0], skip_special_tokens=True)
|
24 |
+
|
25 |
+
print(response)
|
26 |
+
```
|
27 |
+
Evaluation Metrics
|
28 |
+
Model performance was evaluated using Semantic Textual Similarity, Word Mover's Distance, and Grammar Errors. Detailed evaluation results can be found in the accompanying paper.
|
29 |
+
|
30 |
+
Limitations
|
31 |
+
While this model has been fine-tuned on a specific biomedical dataset, it may not perform equally well on other medical or general domain questions. Additionally, the model may generate plausible-sounding but incorrect answers. Always verify the generated answers with reliable sources before using them for critical decision-making.
|
32 |
+
|
33 |
+
Acknowledgements
|
34 |
+
We want to thank Huggingface for their excellent transformers library and the creators of the original DistilGPT2 model. We also thank the authors of the PubMedQA dataset for providing a valuable resource for training and evaluating biomedical question-answering models.
|