metadata

language:
  - nl
license: apache-2.0
base_model: bert-base-uncased
tags:
  - abc
  - generated_from_trainer
datasets:
  - stsb_multi_mt
model-index:
  - name: bert-base-uncased-FinedTuned
    results: []

bert-base-uncased-FinedTuned

This model is a fine-tuned version of bert-base-uncased on the stsb_multi_mt dataset. It achieves the following results on the evaluation set:

Loss: 2.7341
Pearson: 0.2384
Mse: 2.7341
Custom Accuracy: 0.2567
Dataset Accuracy: 0.1762

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 16
seed: 42
gradient_accumulation_steps: 2
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 1000
training_steps: 12000
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Pearson	Mse	Custom Accuracy	Dataset Accuracy
0.0188	5.5556	1000	2.9224	0.2311	2.9224	0.2429	0.1762
0.0367	11.1111	2000	2.8363	0.2219	2.8363	0.2524	0.1762
0.0151	16.6667	3000	2.8033	0.2131	2.8033	0.2509	0.1762
0.0377	22.2222	4000	2.9081	0.2205	2.9081	0.2582	0.1762
0.0458	27.7778	5000	2.8001	0.2360	2.8001	0.2611	0.1762
0.0324	33.3333	6000	2.7521	0.2377	2.7521	0.2567	0.1762
0.0479	38.8889	7000	2.7011	0.2441	2.7011	0.2618	0.1762
0.0685	44.4444	8000	2.7119	0.2431	2.7119	0.2611	0.1762
0.0463	50.0	9000	2.7674	0.2287	2.7674	0.2603	0.1762
0.0879	55.5556	10000	2.7357	0.2434	2.7357	0.2676	0.1762
0.0733	61.1111	11000	2.7392	0.2374	2.7392	0.2567	0.1762
0.1541	66.6667	12000	2.7341	0.2384	2.7341	0.2567	0.1762

Framework versions

Transformers 4.42.3
Pytorch 2.3.1+cu121
Datasets 2.20.0
Tokenizers 0.19.1