mohammadtaghizadeh
commited on
Commit
•
c672dbe
1
Parent(s):
803f62c
Update README.md
Browse files
README.md
CHANGED
@@ -30,20 +30,97 @@ should probably proofread and complete it, then remove this comment. -->
|
|
30 |
|
31 |
# flan-t5-base-imdb-text-classification
|
32 |
|
33 |
-
This model is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) on the imdb dataset.
|
34 |
It achieves the following results on the evaluation set:
|
35 |
- Loss: 0.0767
|
36 |
- F1: 95.084
|
37 |
- Gen Len: 2.4976
|
38 |
|
39 |
-
|
|
|
|
|
|
|
|
|
40 |
|
41 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
|
43 |
|
44 |
## Training and evaluation data
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
45 |
|
46 |
-
More information needed
|
47 |
|
48 |
## Training procedure
|
49 |
|
@@ -60,7 +137,10 @@ The following hyperparameters were used during training:
|
|
60 |
|
61 |
### Training results
|
62 |
|
63 |
-
|
|
|
|
|
|
|
64 |
|
65 |
### Framework versions
|
66 |
|
|
|
30 |
|
31 |
# flan-t5-base-imdb-text-classification
|
32 |
|
33 |
+
This model is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) on the [IMDB](https://huggingface.co/datasets/imdb) dataset.
|
34 |
It achieves the following results on the evaluation set:
|
35 |
- Loss: 0.0767
|
36 |
- F1: 95.084
|
37 |
- Gen Len: 2.4976
|
38 |
|
39 |
+
```
|
40 |
+
precision recall f1-score support
|
41 |
+
|
42 |
+
0 0.97 0.88 0.92 12500
|
43 |
+
1 0.89 0.97 0.93 12500
|
44 |
|
45 |
+
accuracy 0.93 25000
|
46 |
+
macro avg 0.93 0.93 0.93 25000
|
47 |
+
weighted avg 0.93 0.93 0.93 25000
|
48 |
+
```
|
49 |
+
|
50 |
+
## Model description
|
51 |
+
In this implementation, using the **Flan T5 large language model**, we performed the Text Classification task on the IMDB dataset and obtained a very good **accuracy of 93%**.
|
52 |
|
53 |
|
54 |
## Training and evaluation data
|
55 |
+
This model was trained on the imdb train dataset with 25,000 data and then tested and evaluated on the imdb test dataset with 25,000 data.
|
56 |
+
|
57 |
+
## Usage
|
58 |
+
|
59 |
+
- Install dependencies
|
60 |
+
```python
|
61 |
+
!pip install transformers==4.28.1 datasets
|
62 |
+
```
|
63 |
+
|
64 |
+
- Load IMDB Corpus
|
65 |
+
```python
|
66 |
+
from datasets import load_dataset
|
67 |
+
|
68 |
+
dataset_id = "imdb"
|
69 |
+
|
70 |
+
# Load dataset from the hub
|
71 |
+
dataset = load_dataset(dataset_id)
|
72 |
+
```
|
73 |
+
|
74 |
+
- Load fine tune flan t5 model
|
75 |
+
```python
|
76 |
+
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
|
77 |
+
tokenizer = AutoTokenizer.from_pretrained("mohammadtaghizadeh/flan-t5-base-imdb-text-classification")
|
78 |
+
model = AutoModelForSeq2SeqLM.from_pretrained("mohammadtaghizadeh/flan-t5-base-imdb-text-classification")
|
79 |
+
model.to('cuda')
|
80 |
+
```
|
81 |
+
|
82 |
+
- Test the model
|
83 |
+
```python
|
84 |
+
from tqdm.auto import tqdm
|
85 |
+
|
86 |
+
samples_number = len(dataset['test'])
|
87 |
+
progress_bar = tqdm(range(samples_number))
|
88 |
+
predictions_list = []
|
89 |
+
labels_list = []
|
90 |
+
for i in range(samples_number):
|
91 |
+
text = dataset['test']['text'][i]
|
92 |
+
inputs = tokenizer.encode_plus(text, padding='max_length', max_length=512, return_tensors='pt').to('cuda')
|
93 |
+
outputs = model.generate(inputs['input_ids'], attention_mask=inputs['attention_mask'], max_length=150, num_beams=4, early_stopping=True)
|
94 |
+
prediction = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
95 |
+
predictions_list.append(prediction)
|
96 |
+
labels_list.append(dataset['test']['label'][i])
|
97 |
+
|
98 |
+
progress_bar.update(1)
|
99 |
+
```
|
100 |
+
|
101 |
+
- Classification report
|
102 |
+
```python
|
103 |
+
from sklearn.metrics import classification_report
|
104 |
+
|
105 |
+
str_labels_list = []
|
106 |
+
for i in range(len(labels_list)): str_labels_list.append(str(labels_list[i]))
|
107 |
+
|
108 |
+
report = classification_report(str_labels_list, predictions_list)
|
109 |
+
print(report)
|
110 |
+
```
|
111 |
+
|
112 |
+
- Output
|
113 |
+
```cmd
|
114 |
+
precision recall f1-score support
|
115 |
+
|
116 |
+
0 0.97 0.88 0.92 12500
|
117 |
+
1 0.89 0.97 0.93 12500
|
118 |
+
|
119 |
+
accuracy 0.93 25000
|
120 |
+
macro avg 0.93 0.93 0.93 25000
|
121 |
+
weighted avg 0.93 0.93 0.93 25000
|
122 |
+
```
|
123 |
|
|
|
124 |
|
125 |
## Training procedure
|
126 |
|
|
|
137 |
|
138 |
### Training results
|
139 |
|
140 |
+
| Training Loss | Epoch | Step |
|
141 |
+
|:-------------:|:-----:|:----:|
|
142 |
+
| 0.100500 | 1.0 | 3125 |
|
143 |
+
| 0.043600 | 2.0 | 6250 |
|
144 |
|
145 |
### Framework versions
|
146 |
|