Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
language: eng
|
3 |
+
datasets:
|
4 |
+
- banking77
|
5 |
+
---
|
6 |
+
|
7 |
+
# GPT2 Fine-tuned KO
|
8 |
+
This is a fine-tuned version of the GPT2 model. It's best suited for text-generation.
|
9 |
+
|
10 |
+
## Model Description
|
11 |
+
gpt2-finetuned-ko was fine tuned on the [banking77](https://huggingface.co/datasets/banking77) dataset, which is "composed of online banking queries annotated with their corresponding intents."
|
12 |
+
|
13 |
+
## Intended Uses and Limitations
|
14 |
+
Given the hugeness of the [Microsoft DialoGPT-large](https://huggingface.co/microsoft/DialoGPT-large) model, the author resorted to fine-tuning the gpt2 model for the creation of a chatbot. The intent was for the chatbot to emulate a banking customer agent, hence the use of the banking77 dataset. However, when the fine-tuned model was deployed in the chatbot, the results were undesirable. Its responses were inappropriate, unnecessarily long and repetitive. The model performs better in text-generation but is prone to generate baking-related texted because of the corpus it was trained on.
|
15 |
+
|
16 |
+
### How to use
|
17 |
+
|
18 |
+
You can use this model directly with a pipeline for text generation:
|
19 |
+
|
20 |
+
```python
|
21 |
+
>>>from transformers import pipeline
|
22 |
+
|
23 |
+
>>> model_name = "Kwaku/gpt2-finetuned-ko"
|
24 |
+
>>> generator = pipeline("text-generation", model=model_name)
|
25 |
+
>>> result = generator("My money is", max_length=15, num_return_sequences=2)
|
26 |
+
>>> print(result)
|
27 |
+
|
28 |
+
[{'generated_text': 'My money is stuck in ATM pending. Please cancel this transaction and refund it'}, {'generated_text': 'My money is missing. How do I get a second card, and how'}]
|
29 |
+
```
|
30 |
+
|
31 |
+
### Limitations and bias
|
32 |
+
|
33 |
+
For users who want a diverse text-generator, this model's tendency to generate mostly bank-related text will be a drawback. It also inherits [the biases of its parent model, the GPT2](https://huggingface.co/gpt2#limitations-and-bias).
|