cmi-intent-classifier

This model is a fine-tuned version of SI2M-Lab/DarijaBERT on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0656
• Accuracy: {'accuracy': 0.9856670341786108}
• Precision: {'precision': 0.9858055469318123}
• F1: {'f1': 0.9856943230568389}
• Recall: {'recall': 0.9856670341786108}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 16

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	F1	Recall
No log	1.0	227	0.0775	{'accuracy': 0.9735391400220507}	{'precision': 0.9757984503504595}	{'f1': 0.973790037472012}	{'recall': 0.9735391400220507}
No log	2.0	454	0.0725	{'accuracy': 0.9812568908489526}	{'precision': 0.9821967937934353}	{'f1': 0.9813050908586485}	{'recall': 0.9812568908489526}
0.2168	3.0	681	0.0670	{'accuracy': 0.9845644983461963}	{'precision': 0.9847236892573811}	{'f1': 0.9845765826750075}	{'recall': 0.9845644983461963}
0.2168	4.0	908	0.0656	{'accuracy': 0.9856670341786108}	{'precision': 0.9858055469318123}	{'f1': 0.9856943230568389}	{'recall': 0.9856670341786108}
0.0067	5.0	1135	0.0916	{'accuracy': 0.9812568908489526}	{'precision': 0.9815359056460955}	{'f1': 0.9812950855976162}	{'recall': 0.9812568908489526}
0.0067	6.0	1362	0.0759	{'accuracy': 0.9856670341786108}	{'precision': 0.9857960025524459}	{'f1': 0.985690448418444}	{'recall': 0.9856670341786108}
0.0014	7.0	1589	0.0985	{'accuracy': 0.9812568908489526}	{'precision': 0.9815646078321539}	{'f1': 0.981280504019021}	{'recall': 0.9812568908489526}
0.0014	8.0	1816	0.0777	{'accuracy': 0.9823594266813671}	{'precision': 0.982518298195663}	{'f1': 0.9823858677111299}	{'recall': 0.9823594266813671}
0.0015	9.0	2043	0.0770	{'accuracy': 0.9823594266813671}	{'precision': 0.9825331363623647}	{'f1': 0.9823970674366541}	{'recall': 0.9823594266813671}
0.0015	10.0	2270	0.0683	{'accuracy': 0.9812568908489526}	{'precision': 0.9814500745020152}	{'f1': 0.9812964736205629}	{'recall': 0.9812568908489526}
0.0015	11.0	2497	0.0688	{'accuracy': 0.9823594266813671}	{'precision': 0.982518298195663}	{'f1': 0.9823858677111299}	{'recall': 0.9823594266813671}
0.0002	12.0	2724	0.0714	{'accuracy': 0.9823594266813671}	{'precision': 0.982518298195663}	{'f1': 0.9823858677111299}	{'recall': 0.9823594266813671}
0.0002	13.0	2951	0.0662	{'accuracy': 0.9823594266813671}	{'precision': 0.982518298195663}	{'f1': 0.9823858677111299}	{'recall': 0.9823594266813671}
0.0002	14.0	3178	0.0674	{'accuracy': 0.9834619625137817}	{'precision': 0.9836807898398305}	{'f1': 0.9834925574843851}	{'recall': 0.9834619625137817}
0.0002	15.0	3405	0.0682	{'accuracy': 0.9834619625137817}	{'precision': 0.9836807898398305}	{'f1': 0.9834925574843851}	{'recall': 0.9834619625137817}
0.0002	16.0	3632	0.0684	{'accuracy': 0.9834619625137817}	{'precision': 0.9836807898398305}	{'f1': 0.9834925574843851}	{'recall': 0.9834619625137817}

Framework versions

• Transformers 4.38.0.dev0
• Pytorch 2.1.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2