Korean GPT Bot Sentiment Classification (ko-gpt-bot-sc)
Method
- Promt-Tuning/Prefix-tuning/Soft Embedding
- Parameters
| Parameters |
No. |
| All |
6173039616 (100.0%) |
| Trainable |
6537216 (0.1%) |
| Freezed |
6166502400 (99.9%) |
Model
LAYER NAME #PARAMS RATIO MEM(MB)
--model: 6,177,233,921 100.00% 23552.28
--learned_embedding: 6,537,216 0.11% 24.94
--transformer: 5,906,391,041 95.62% 22519.09
--wte
--weight: 264,241,152 4.28% 1008.00
--h: 5,642,141,697 91.34% 21511.06
--0: 205,549,569 3.33% 772.11
--ln_1: 8,192 0.00% 0.03
--attn: 71,303,169 1.15% 260.00
--mlp: 134,238,208 2.17% 512.08
--1(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--2(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--3(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--4(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--5(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--6(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--7(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--8(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--9(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--10(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--11(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--12(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--13(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--14(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--15(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--16(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--17(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--18(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--19(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--20(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--21(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--22(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--23(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--24(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--25(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--26(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--27(partially shared): 201,355,264 3.26% 768.11
--ln_1: 8,192 0.00% 0.03
--attn(shared): 67,108,864 1.09% 256.00
--mlp: 134,238,208 2.17% 512.08
--ln_f: 8,192 0.00% 0.03
--weight: 4,096 0.00% 0.02
--bias: 4,096 0.00% 0.02
--lm_head: 264,305,664 4.28% 1008.25
--weight: 264,241,152 4.28% 1008.00
--bias: 64,512 0.00% 0.25
Metrics
| Metric |
Value |
| step |
520 |
| loss |
3.1814 |
|
precision |
recall |
f1-score |
support |
| 긍정 |
0.92549 |
0.944 |
0.934653 |
500 |
| 부정 |
0.942857 |
0.924 |
0.933333 |
500 |
| accuracy |
0.934 |
0.934 |
0.934 |
0.934 |
| macro avg |
0.934174 |
0.934 |
0.933993 |
1000 |
| weighted avg |
0.934174 |
0.934 |
0.933993 |
1000 |
References
