iati-climate-multi-classifier-weighted2
This model is a fine-tuned version of alex-miller/ODABert on a subset of the alex-miller/iati-policy-markers dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7080
- Accuracy: 0.8541
- F1: 0.7121
- Precision: 0.6265
- Recall: 0.8248
Model description
This model has been trained to identify both significant and principal climate mitigation and climate adaptation project titles and/or descriptions.
Intended uses & limitations
As many of the donors in the training dataset have mixed up Adaptation and Mitigation, the model's ability to differentiate the two isn't perfect. But the sigmoid of the model logits do bias toward the correct class.
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-06
- train_batch_size: 24
- eval_batch_size: 24
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20
Training results
Training Loss | Epoch | Step | Accuracy | F1 | Validation Loss | Precision | Recall |
---|---|---|---|---|---|---|---|
0.7689 | 1.0 | 1951 | 0.7993 | 0.6421 | 0.6477 | 0.5264 | 0.8230 |
0.6217 | 2.0 | 3902 | 0.8303 | 0.6737 | 0.6269 | 0.5814 | 0.8010 |
0.5834 | 3.0 | 5853 | 0.8266 | 0.6761 | 0.6101 | 0.5715 | 0.8276 |
0.5571 | 4.0 | 7804 | 0.8461 | 0.6933 | 0.6169 | 0.6144 | 0.7954 |
0.5323 | 5.0 | 9755 | 0.8366 | 0.6869 | 0.6050 | 0.5913 | 0.8194 |
0.5126 | 6.0 | 11706 | 0.8327 | 0.6867 | 0.6047 | 0.5815 | 0.8385 |
0.4968 | 7.0 | 13657 | 0.8408 | 0.6938 | 0.6098 | 0.5989 | 0.8244 |
0.4893 | 8.0 | 15608 | 0.6040 | 0.8348 | 0.6895 | 0.5854 | 0.8387 |
0.4702 | 9.0 | 17559 | 0.6342 | 0.8508 | 0.7050 | 0.6211 | 0.8151 |
0.4514 | 10.0 | 19510 | 0.6210 | 0.8383 | 0.6946 | 0.5918 | 0.8404 |
0.4323 | 11.0 | 21461 | 0.6340 | 0.8402 | 0.6991 | 0.5943 | 0.8487 |
0.4193 | 12.0 | 23412 | 0.6407 | 0.8433 | 0.7005 | 0.6020 | 0.8375 |
0.407 | 13.0 | 25363 | 0.6602 | 0.8526 | 0.7094 | 0.6237 | 0.8223 |
0.3944 | 14.0 | 27314 | 0.6588 | 0.8441 | 0.7026 | 0.6029 | 0.8419 |
0.3834 | 15.0 | 29265 | 0.6881 | 0.8529 | 0.7110 | 0.6233 | 0.8274 |
0.3738 | 16.0 | 31216 | 0.7029 | 0.8575 | 0.7146 | 0.6359 | 0.8155 |
0.3686 | 17.0 | 33167 | 0.6929 | 0.8524 | 0.7102 | 0.6224 | 0.8271 |
0.3607 | 18.0 | 35118 | 0.7069 | 0.8545 | 0.7127 | 0.6272 | 0.8253 |
0.3556 | 19.0 | 37069 | 0.7072 | 0.8543 | 0.7118 | 0.6274 | 0.8225 |
0.3523 | 20.0 | 39020 | 0.7080 | 0.8541 | 0.7121 | 0.6265 | 0.8248 |
Framework versions
- Transformers 4.41.0
- Pytorch 2.0.1
- Datasets 2.19.1
- Tokenizers 0.19.1
- Downloads last month
- 26
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for alex-miller/iati-climate-multi-classifier-weighted2
Base model
google-bert/bert-base-multilingual-uncased
Finetuned
alex-miller/ODABert
Finetuned
this model