metadata
language:
- ko
Model description
Intent
- 의도없음
- 조명전원켜짐
- 조명전원꺼짐
- 조명밝게설정
- 조명어둡게설정
- 조명색상설정
- 조명밝기조절
- 조명색상모드설정
- 조명차갑게설정
- 조명따뜻하게설정
- 모드실행
- 상태조회
- 조명전체켜짐
- 조명전체꺼짐
- 조명켜짐예약등록
- 조명꺼짐예약등록
- 조명예약취소
- 별칭등록
Slots
- O
- mode
- time
- home
- device
- nickname
- percent
- color_red
- color_yellow
- color_blue
- color_mode
Training hyperparameters
The following hyperparameters were used during training:
- train_batch_size: 32
- eval_batch_size: 32
- gradient_accumulation_steps: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=2.0e-05
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 10
- weight_decay: 0.01
- intent_loss_weight: 0.6
Evaluation results
Intent report:
label precision recall f1 support
의도없음 (label_id: 0) 83.33 100.00 90.91 5
조명전원켜짐 (label_id: 1) 100.00 100.00 100.00 10
조명전원꺼짐 (label_id: 2) 100.00 100.00 100.00 10
조명밝게설정 (label_id: 3) 76.92 100.00 86.96 10
조명어둡게설정 (label_id: 4) 100.00 100.00 100.00 10
조명색상설정 (label_id: 5) 83.33 100.00 90.91 10
조명밝기조절 (label_id: 6) 100.00 100.00 100.00 10
조명색상모드설정 (label_id: 7) 100.00 100.00 100.00 10
조명차갑게설정 (label_id: 8) 90.00 100.00 94.74 9
조명따뜻하게설정 (label_id: 9) 100.00 90.91 95.24 11
모드실행 (label_id: 10) 100.00 100.00 100.00 10
상태조회 (label_id: 11) 100.00 80.00 88.89 10
조명전체켜짐 (label_id: 12) 100.00 100.00 100.00 10
조명전체꺼짐 (label_id: 13) 100.00 100.00 100.00 10
조명켜짐예약등록 (label_id: 14) 100.00 100.00 100.00 10
조명꺼짐예약등록 (label_id: 15) 90.91 100.00 95.24 10
조명예약취소 (label_id: 16) 100.00 90.00 94.74 10
별칭등록 (label_id: 17) 100.00 60.00 75.00 10
-------------------
micro avg 95.43 95.43 95.43 175
macro avg 95.81 95.61 95.15 175
weighted avg 96.22 95.43 95.27 175
Slot report:
label precision recall f1 support
O (label_id: 0) 95.16 89.29 92.13 308
mode (label_id: 1) 90.91 100.00 95.24 10
time (label_id: 2) 95.74 100.00 97.83 45
home (label_id: 3) 91.85 96.12 93.94 129
device (label_id: 4) 84.62 92.26 88.27 155
nickname (label_id: 5) 100.00 42.86 60.00 7
percent (label_id: 6) 100.00 100.00 100.00 10
color_red (label_id: 7) 80.00 100.00 88.89 4
color_yellow (label_id: 8) 100.00 100.00 100.00 3
color_blue (label_id: 9) 100.00 100.00 100.00 2
color_mode (label_id: 10) 100.00 90.00 94.74 10
-------------------
micro avg 91.95 91.95 91.95 683
macro avg 94.39 91.87 91.91 683
weighted avg 92.25 91.95 91.88 683