metadata

language:
  - nl
license: apache-2.0
base_model: bert-base-uncased
tags:
  - abc
  - generated_from_trainer
datasets:
  - stsb_multi_mt
model-index:
  - name: bert-base-uncased-FinedTuned
    results: []

bert-base-uncased-FinedTuned

This model is a fine-tuned version of bert-base-uncased on the stsb_multi_mt dataset. It achieves the following results on the evaluation set:

Loss: 2.7638
Pearson: 0.2339
Mse: 2.7638
Custom Accuracy: 0.2603
Dataset Accuracy: 0.1762

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 1000
training_steps: 12000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Pearson	Mse	Custom Accuracy	Dataset Accuracy
0.0219	5.5556	1000	2.8140	0.2324	2.8140	0.2437	0.1762
0.0195	11.1111	2000	2.9679	0.2078	2.9679	0.2618	0.1762
0.0184	16.6667	3000	2.7712	0.2476	2.7712	0.2683	0.1762
0.0213	22.2222	4000	2.7564	0.2486	2.7564	0.2661	0.1762
0.0222	27.7778	5000	2.8691	0.2333	2.8691	0.2596	0.1762
0.0151	33.3333	6000	2.7762	0.2451	2.7762	0.2560	0.1762
0.0318	38.8889	7000	2.8121	0.2370	2.8121	0.2647	0.1762
0.0616	44.4444	8000	2.8343	0.2195	2.8343	0.2560	0.1762
0.0335	50.0	9000	2.8070	0.2259	2.8070	0.2676	0.1762
0.0553	55.5556	10000	2.7934	0.2330	2.7934	0.2531	0.1762
0.0718	61.1111	11000	2.7822	0.2286	2.7822	0.2603	0.1762
0.1741	66.6667	12000	2.7638	0.2339	2.7638	0.2603	0.1762

Framework versions

Transformers 4.42.3
Pytorch 2.3.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1