Hugging Face's logo

lygitdata
/
garmentiq

ONNX
Model card Files
xet
 Community
lygitdata commited on
Commit
5f02016
·
verified ·
1 Parent(s): 850f46a

Create classification_model_comparison.md

Browse files
Files changed (1) hide show
  1. classification_model_comparison.md +183 -0
classification_model_comparison.md ADDED
@@ -0,0 +1,183 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Model fine-tuning briefing
2
+
3
+ Date: 06/08/2025
4
+
5
+ > For `.pt` model files, please go to https://huggingface.co/lygitdata/garmentiq.
6
+
7
+ Ranking:
8
+
9
+ - 🏆 1st place: tinyViT (`tiny_vit_inditex_finetuned.pt`) - execellent performance on new Zara data (99.16% accuracy), preserves strong performance on the original Nordstrom & Myntra testing data (94.84% accuracy).
10
+
11
+ - 🥈 2nd place: CNN4 (`cnn_4_inditex_finetuned.pt`)
12
+
13
+ - 🥉 3rd place: CNN3 (`cnn_3_inditex_finetuned.pt`)
14
+
15
+ *Note*: All these models were fine-tuned using optimizer `AdamW` with learning rate `0.00001` and weight decay `0.0001`.
16
+
17
+ Data:
18
+
19
+ - Zara data: https://www.kaggle.com/datasets/lygitdata/zara-clothes-image-data
20
+
21
+ - Nordstrom & Myntra data: https://www.kaggle.com/datasets/lygitdata/garmentiq-classification-set-nordstrom-and-myntra
22
+
23
+ ---
24
+
25
+ ## tinyViT performance with fine-tuning
26
+
27
+ ### Training evaluation on Zara data
28
+
29
+ ```
30
+ Test Loss: 0.0390
31
+ Test Accuracy: 0.9916
32
+ Test F1 Score: 0.9917
33
+
34
+ Classification Report:
35
+ precision recall f1-score support
36
+
37
+ long sleeve dress 1.00 0.99 0.99 80
38
+ long sleeve top 0.99 0.99 0.99 352
39
+ short sleeve dress 0.93 1.00 0.97 14
40
+ short sleeve top 1.00 1.00 1.00 84
41
+ shorts 1.00 1.00 1.00 56
42
+ skirt 1.00 1.00 1.00 62
43
+ trousers 1.00 1.00 1.00 80
44
+ vest 0.96 0.98 0.97 94
45
+ vest dress 1.00 1.00 1.00 12
46
+
47
+ accuracy 0.99 834
48
+ macro avg 0.99 0.99 0.99 834
49
+ weighted avg 0.99 0.99 0.99 834
50
+ ```
51
+
52
+ ### Test on original Nordstrom & Myntra testing data
53
+
54
+ ```
55
+ Test Loss: 0.1683
56
+ Test Accuracy: 0.9484
57
+ Test F1 Score: 0.9483
58
+
59
+ Classification Report:
60
+ precision recall f1-score support
61
+
62
+ long sleeve dress 0.89 0.95 0.92 384
63
+ long sleeve top 0.94 1.00 0.97 442
64
+ short sleeve dress 0.90 0.88 0.89 382
65
+ short sleeve top 0.98 0.98 0.98 523
66
+ shorts 0.99 0.97 0.98 485
67
+ skirt 0.98 0.90 0.94 281
68
+ trousers 0.97 0.99 0.98 320
69
+ vest 0.95 0.89 0.92 230
70
+ vest dress 0.94 0.93 0.93 442
71
+
72
+ accuracy 0.95 3489
73
+ macro avg 0.95 0.94 0.95 3489
74
+ weighted avg 0.95 0.95 0.95 3489
75
+ ```
76
+
77
+ ---
78
+
79
+ ## CNN4 performance with fine-tuning
80
+
81
+ ### Training evaluation on Zara data
82
+
83
+ ```
84
+ Test Loss: 0.1355
85
+ Test Accuracy: 0.9592
86
+ Test F1 Score: 0.9585
87
+
88
+ Classification Report:
89
+ precision recall f1-score support
90
+
91
+ long sleeve dress 0.98 0.99 0.98 80
92
+ long sleeve top 0.99 0.97 0.98 352
93
+ short sleeve dress 0.75 0.86 0.80 14
94
+ short sleeve top 0.92 0.98 0.95 84
95
+ shorts 0.96 0.88 0.92 56
96
+ skirt 0.88 0.97 0.92 62
97
+ trousers 0.98 1.00 0.99 80
98
+ vest 0.97 0.97 0.97 94
99
+ vest dress 0.86 0.50 0.63 12
100
+
101
+ accuracy 0.96 834
102
+ macro avg 0.92 0.90 0.90 834
103
+ weighted avg 0.96 0.96 0.96 834
104
+ ```
105
+
106
+ ### Test on original Nordstrom & Myntra testing data
107
+
108
+ ```
109
+ Test Loss: 0.3326
110
+ Test Accuracy: 0.9132
111
+ Test F1 Score: 0.9137
112
+
113
+ Classification Report:
114
+ precision recall f1-score support
115
+
116
+ long sleeve dress 0.92 0.83 0.87 384
117
+ long sleeve top 0.97 0.88 0.92 442
118
+ short sleeve dress 0.75 0.97 0.84 382
119
+ short sleeve top 0.91 0.97 0.94 523
120
+ shorts 0.94 0.98 0.96 485
121
+ skirt 0.95 0.83 0.88 281
122
+ trousers 0.96 0.97 0.97 320
123
+ vest 0.95 0.85 0.90 230
124
+ vest dress 0.94 0.88 0.91 442
125
+
126
+ accuracy 0.91 3489
127
+ macro avg 0.92 0.91 0.91 3489
128
+ weighted avg 0.92 0.91 0.91 3489
129
+ ```
130
+
131
+ ---
132
+
133
+ ## CNN3 performance with fine-tuning
134
+
135
+ ### Training evaluation on Zara data
136
+
137
+ ```
138
+ Test Loss: 0.2756
139
+ Test Accuracy: 0.9197
140
+ Test F1 Score: 0.9216
141
+
142
+ Classification Report:
143
+ precision recall f1-score support
144
+
145
+ long sleeve dress 0.89 0.95 0.92 80
146
+ long sleeve top 0.97 0.95 0.96 352
147
+ short sleeve dress 0.62 0.71 0.67 14
148
+ short sleeve top 0.94 0.95 0.95 84
149
+ shorts 0.94 0.86 0.90 56
150
+ skirt 0.79 0.92 0.85 62
151
+ trousers 0.99 1.00 0.99 80
152
+ vest 0.92 0.82 0.87 94
153
+ vest dress 0.35 0.50 0.41 12
154
+
155
+ accuracy 0.92 834
156
+ macro avg 0.82 0.85 0.83 834
157
+ weighted avg 0.93 0.92 0.92 834
158
+ ```
159
+
160
+ ### Test on original Nordstrom & Myntra testing data
161
+
162
+ ```
163
+ Test Loss: 0.3338
164
+ Test Accuracy: 0.9074
165
+ Test F1 Score: 0.9068
166
+
167
+ Classification Report:
168
+ precision recall f1-score support
169
+
170
+ long sleeve dress 0.84 0.89 0.86 384
171
+ long sleeve top 0.94 0.93 0.94 442
172
+ short sleeve dress 0.76 0.94 0.84 382
173
+ short sleeve top 0.93 0.98 0.95 523
174
+ shorts 0.96 0.97 0.96 485
175
+ skirt 0.94 0.81 0.87 281
176
+ trousers 0.94 0.99 0.97 320
177
+ vest 0.99 0.69 0.81 230
178
+ vest dress 0.94 0.84 0.89 442
179
+
180
+ accuracy 0.91 3489
181
+ macro avg 0.92 0.89 0.90 3489
182
+ weighted avg 0.91 0.91 0.91 3489
183
+ ```