metadata

license: apache-2.0
base_model: line-corporation/line-distilbert-base-japanese
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: fc-binary-model
    results: []

fc-binary-model

This model is a fine-tuned version of line-corporation/line-distilbert-base-japanese on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.3003
Accuracy: 0.8730

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 64
eval_batch_size: 8
seed: 42
distributed_type: tpu
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	306	0.3749	0.8594
0.4137	2.0	612	0.3596	0.8594
0.4137	3.0	918	0.3459	0.8594
0.383	4.0	1224	0.3423	0.8613
0.3709	5.0	1530	0.3348	0.8613
0.3709	6.0	1836	0.3292	0.8672
0.364	7.0	2142	0.3275	0.8633
0.364	8.0	2448	0.3235	0.8652
0.3587	9.0	2754	0.3227	0.8633
0.3509	10.0	3060	0.3182	0.8652
0.3509	11.0	3366	0.3154	0.8730
0.3531	12.0	3672	0.3132	0.8691
0.3531	13.0	3978	0.3108	0.8691
0.3478	14.0	4284	0.3112	0.875
0.344	15.0	4590	0.3086	0.8711
0.344	16.0	4896	0.3070	0.8730
0.3386	17.0	5202	0.3047	0.875
0.34	18.0	5508	0.3058	0.875
0.34	19.0	5814	0.3054	0.875
0.3356	20.0	6120	0.3043	0.8730
0.3356	21.0	6426	0.3037	0.8770
0.3331	22.0	6732	0.3034	0.875
0.3356	23.0	7038	0.3022	0.875
0.3356	24.0	7344	0.3019	0.8730
0.3317	25.0	7650	0.3022	0.8711
0.3317	26.0	7956	0.3017	0.8711
0.3275	27.0	8262	0.3011	0.8770
0.328	28.0	8568	0.3005	0.8730
0.328	29.0	8874	0.3004	0.8730
0.3315	30.0	9180	0.3003	0.8730

Framework versions

Transformers 4.34.0
Pytorch 2.0.0+cu118
Datasets 2.14.5
Tokenizers 0.14.0