Luna-Skywalker/BERT-crime-analysis

Overview

This model is a fine-tuned version of google-bert/bert-base-uncased on this Kaggle dataset. It achieves the following results on the evaluation set:

• Macro f1: 89.44%
• Weighted f1: 93.15%
• Accuracy: 93.80%
• Balanced accuracy: 90.42%

Model description

This finetuned version of google-bert/bert-base-uncased excels at detecting the crime type from the description of the crime. It has 34 labels.

Training and evaluation data

• eval_macro f1: 89.44%
• eval_weighted f1: 93.15%
• eval_accuracy: 93.79%
• eval_balanced accuracy: 90.42%

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Macro f1	Weighted f1	Accuracy	Balanced accuracy
0.1859	1.0	5538	0.1297	0.8561	0.9249	0.9366	0.8571
0.1281	2.0	11076	0.1260	0.8702	0.9248	0.9369	0.8740
0.1279	3.0	16614	0.1251	0.8728	0.9314	0.9380	0.8749
0.1272	4.0	22152	0.1276	0.8652	0.9247	0.9367	0.8655
0.1266	5.0	27690	0.1256	0.8685	0.9252	0.9345	0.8724
0.1284	6.0	33228	0.1264	0.8668	0.9252	0.9345	0.8724
0.1272	7.0	38766	0.1247	0.8739	0.9313	0.9379	0.8748
0.1262	8.0	44304	0.1258	0.8892	0.9246	0.9366	0.9024
0.1263	9.0	49842	0.1251	0.9038	0.9310	0.9378	0.9041
0.1267	10.0	55380	0.1244	0.8897	0.9253	0.9345	0.9018
0.1271	11.0	60918	0.1251	0.8951	0.9325	0.9371	0.9036
0.1268	12.0	66456	0.1248	0.8944	0.9315	0.9380	0.9042
0.1254	13.0	71994	0.1247	0.9038	0.9314	0.9381	0.9043
0.126	14.0	77532	0.1263	0.8944	0.9314	0.9379	0.9042
0.1261	15.0	83070	0.1274	0.8891	0.9250	0.9348	0.9020
0.1253	16.0	88608	0.1241	0.8944	0.9315	0.9380	0.9042
0.1251	17.0	94146	0.1244	0.9042	0.9314	0.9380	0.9042
0.125	18.0	99684	0.1249	0.9041	0.9314	0.9380	0.9043
0.125	19.0	105222	0.1245	0.8942	0.9312	0.9380	0.9042
0.1257	20.0	110760	0.1248	0.9041	0.9313	0.9379	0.9042
0.125	21.0	116298	0.1248	0.9000	0.9254	0.9344	0.9018
0.1248	22.0	121836	0.1244	0.9041	0.9313	0.9379	0.9042
0.1246	23.0	127374	0.1245	0.9042	0.9315	0.9380	0.9042
0.1247	24.0	132912	0.1242	0.8943	0.9314	0.9380	0.9043
0.1245	25.0	138450	0.1242	0.9042	0.9315	0.9380	0.9042
0.1245	26.0	143988	0.1245	0.9042	0.9314	0.9381	0.9043
0.1245	27.0	149526	0.1242	0.8944	0.9314	0.9381	0.9043
0.1244	28.0	155064	0.1242	0.9336	0.9315	0.9381	0.9337
0.1243	29.0	160602	0.1243	0.8944	0.9314	0.9381	0.9043
0.1243	30.0	166140	0.1243	0.8944	0.9314	0.9381	0.9043

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2
• Datasets 2.18.0
• Tokenizers 0.15.2