city_model

This model is a fine-tuned version of distilbert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0236

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	2	5.5208
No log	2.0	4	5.0539
No log	3.0	6	4.5135
No log	4.0	8	3.9938
No log	5.0	10	3.5662
No log	6.0	12	3.1228
No log	7.0	14	2.6305
No log	8.0	16	2.1844
No log	9.0	18	1.7503
No log	10.0	20	1.3209
No log	11.0	22	0.9585
No log	12.0	24	0.6409
No log	13.0	26	0.4198
No log	14.0	28	0.2886
No log	15.0	30	0.1990
No log	16.0	32	0.1480
No log	17.0	34	0.1210
No log	18.0	36	0.0992
No log	19.0	38	0.0835
No log	20.0	40	0.0953
No log	21.0	42	0.1331
No log	22.0	44	0.1893
No log	23.0	46	0.0807
No log	24.0	48	0.0467
No log	25.0	50	0.0362
No log	26.0	52	0.0329
No log	27.0	54	0.0318
No log	28.0	56	0.0306
No log	29.0	58	0.0274
No log	30.0	60	0.0244
No log	31.0	62	0.0238
No log	32.0	64	0.0220
No log	33.0	66	0.0190
No log	34.0	68	0.0182
No log	35.0	70	0.0180
No log	36.0	72	0.0174
No log	37.0	74	0.0173
No log	38.0	76	0.0183
No log	39.0	78	0.0199
No log	40.0	80	0.0218
No log	41.0	82	0.0225
No log	42.0	84	0.0216
No log	43.0	86	0.0210
No log	44.0	88	0.0209
No log	45.0	90	0.0210
No log	46.0	92	0.0214
No log	47.0	94	0.0223
No log	48.0	96	0.0231
No log	49.0	98	0.0235
No log	50.0	100	0.0236

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0