README.md

metadata

language:
  - nl
license: apache-2.0
base_model: bert-base-uncased
tags:
  - abc
  - generated_from_trainer
datasets:
  - stsb_multi_mt
model-index:
  - name: bert-base-uncased-FinedTuned
    results: []

bert-base-uncased-FinedTuned

This model is a fine-tuned version of bert-base-uncased on the stsb_multi_mt dataset. It achieves the following results on the evaluation set:

• Loss: 2.7490
• Pearson: 0.2351
• Mse: 2.7490
• Custom Accuracy: 0.2647
• Dataset Accuracy: 0.1762

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 12000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Pearson	Mse	Custom Accuracy	Dataset Accuracy
0.0163	5.5556	1000	2.7976	0.2458	2.7976	0.2473	0.1762
0.0163	11.1111	2000	2.9602	0.2203	2.9602	0.2480	0.1762
0.0141	16.6667	3000	2.8549	0.2317	2.8549	0.2647	0.1762
0.0218	22.2222	4000	2.8754	0.2075	2.8754	0.2625	0.1762
0.0061	27.7778	5000	2.8724	0.2360	2.8724	0.2683	0.1762
0.0747	33.3333	6000	2.8425	0.2218	2.8425	0.2516	0.1762
0.0291	38.8889	7000	2.8143	0.2266	2.8143	0.2618	0.1762
0.0973	44.4444	8000	2.7617	0.2327	2.7617	0.2647	0.1762
0.0575	50.0	9000	2.7532	0.2381	2.7532	0.2654	0.1762
0.0717	55.5556	10000	2.8212	0.2249	2.8212	0.2603	0.1762
0.0862	61.1111	11000	2.7608	0.2334	2.7608	0.2647	0.1762
0.1598	66.6667	12000	2.7490	0.2351	2.7490	0.2647	0.1762

Framework versions

• Transformers 4.42.3
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1