mohammadtaghizadeh commited on
Commit
c672dbe
1 Parent(s): 803f62c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +85 -5
README.md CHANGED
@@ -30,20 +30,97 @@ should probably proofread and complete it, then remove this comment. -->
30
 
31
  # flan-t5-base-imdb-text-classification
32
 
33
- This model is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) on the imdb dataset.
34
  It achieves the following results on the evaluation set:
35
  - Loss: 0.0767
36
  - F1: 95.084
37
  - Gen Len: 2.4976
38
 
39
- ## Model description
 
 
 
 
40
 
41
- More information needed
 
 
 
 
 
 
42
 
43
 
44
  ## Training and evaluation data
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
 
46
- More information needed
47
 
48
  ## Training procedure
49
 
@@ -60,7 +137,10 @@ The following hyperparameters were used during training:
60
 
61
  ### Training results
62
 
63
-
 
 
 
64
 
65
  ### Framework versions
66
 
 
30
 
31
  # flan-t5-base-imdb-text-classification
32
 
33
+ This model is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) on the [IMDB](https://huggingface.co/datasets/imdb) dataset.
34
  It achieves the following results on the evaluation set:
35
  - Loss: 0.0767
36
  - F1: 95.084
37
  - Gen Len: 2.4976
38
 
39
+ ```
40
+ precision recall f1-score support
41
+
42
+ 0 0.97 0.88 0.92 12500
43
+ 1 0.89 0.97 0.93 12500
44
 
45
+ accuracy 0.93 25000
46
+ macro avg 0.93 0.93 0.93 25000
47
+ weighted avg 0.93 0.93 0.93 25000
48
+ ```
49
+
50
+ ## Model description
51
+ In this implementation, using the **Flan T5 large language model**, we performed the Text Classification task on the IMDB dataset and obtained a very good **accuracy of 93%**.
52
 
53
 
54
  ## Training and evaluation data
55
+ This model was trained on the imdb train dataset with 25,000 data and then tested and evaluated on the imdb test dataset with 25,000 data.
56
+
57
+ ## Usage
58
+
59
+ - Install dependencies
60
+ ```python
61
+ !pip install transformers==4.28.1 datasets
62
+ ```
63
+
64
+ - Load IMDB Corpus
65
+ ```python
66
+ from datasets import load_dataset
67
+
68
+ dataset_id = "imdb"
69
+
70
+ # Load dataset from the hub
71
+ dataset = load_dataset(dataset_id)
72
+ ```
73
+
74
+ - Load fine tune flan t5 model
75
+ ```python
76
+ from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
77
+ tokenizer = AutoTokenizer.from_pretrained("mohammadtaghizadeh/flan-t5-base-imdb-text-classification")
78
+ model = AutoModelForSeq2SeqLM.from_pretrained("mohammadtaghizadeh/flan-t5-base-imdb-text-classification")
79
+ model.to('cuda')
80
+ ```
81
+
82
+ - Test the model
83
+ ```python
84
+ from tqdm.auto import tqdm
85
+
86
+ samples_number = len(dataset['test'])
87
+ progress_bar = tqdm(range(samples_number))
88
+ predictions_list = []
89
+ labels_list = []
90
+ for i in range(samples_number):
91
+ text = dataset['test']['text'][i]
92
+ inputs = tokenizer.encode_plus(text, padding='max_length', max_length=512, return_tensors='pt').to('cuda')
93
+ outputs = model.generate(inputs['input_ids'], attention_mask=inputs['attention_mask'], max_length=150, num_beams=4, early_stopping=True)
94
+ prediction = tokenizer.decode(outputs[0], skip_special_tokens=True)
95
+ predictions_list.append(prediction)
96
+ labels_list.append(dataset['test']['label'][i])
97
+
98
+ progress_bar.update(1)
99
+ ```
100
+
101
+ - Classification report
102
+ ```python
103
+ from sklearn.metrics import classification_report
104
+
105
+ str_labels_list = []
106
+ for i in range(len(labels_list)): str_labels_list.append(str(labels_list[i]))
107
+
108
+ report = classification_report(str_labels_list, predictions_list)
109
+ print(report)
110
+ ```
111
+
112
+ - Output
113
+ ```cmd
114
+ precision recall f1-score support
115
+
116
+ 0 0.97 0.88 0.92 12500
117
+ 1 0.89 0.97 0.93 12500
118
+
119
+ accuracy 0.93 25000
120
+ macro avg 0.93 0.93 0.93 25000
121
+ weighted avg 0.93 0.93 0.93 25000
122
+ ```
123
 
 
124
 
125
  ## Training procedure
126
 
 
137
 
138
  ### Training results
139
 
140
+ | Training Loss | Epoch | Step |
141
+ |:-------------:|:-----:|:----:|
142
+ | 0.100500 | 1.0 | 3125 |
143
+ | 0.043600 | 2.0 | 6250 |
144
 
145
  ### Framework versions
146