Hugging Face's logo

AntoineD
/
camembert_causal_language_modeling_tools

Text Generation
Transformers
PyTorch
camembert
Generated from Trainer
Inference Endpoints
Model card Files Community
camembert_causal_language_modeling_tools / README.md
AntoineD's picture
AntoineD
End of training
702c710
preview code
|
raw
history blame
No virus
2.8 kB
metadata
license: mit
base_model: camembert-base
tags:
  - generated_from_trainer
model-index:
  - name: camembert_causal_language_modeling_tools
    results: []

camembert_causal_language_modeling_tools

This model is a fine-tuned version of camembert-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0064

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 24
  • eval_batch_size: 192
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss
No log 1.0 5 2.8497
No log 2.0 10 2.0819
No log 3.0 15 1.7468
No log 4.0 20 1.2522
No log 5.0 25 0.6961
No log 6.0 30 0.2864
No log 7.0 35 0.1121
No log 8.0 40 0.0688
No log 9.0 45 0.0579
No log 10.0 50 0.0459
No log 11.0 55 0.0411
No log 12.0 60 0.0382
No log 13.0 65 0.0345
No log 14.0 70 0.0289
No log 15.0 75 0.0228
No log 16.0 80 0.0175
No log 17.0 85 0.0134
No log 18.0 90 0.0130
No log 19.0 95 0.0130
No log 20.0 100 0.0104
No log 21.0 105 0.0096
No log 22.0 110 0.0084
No log 23.0 115 0.0075
No log 24.0 120 0.0065
No log 25.0 125 0.0067
No log 26.0 130 0.0068
No log 27.0 135 0.0069
No log 28.0 140 0.0066
No log 29.0 145 0.0066
No log 30.0 150 0.0064

Framework versions

  • Transformers 4.34.0
  • Pytorch 2.0.1+cu117
  • Datasets 2.14.5
  • Tokenizers 0.14.1