Hugging Face's logo

alex-miller
/
iati-climate-multi-classifier-weighted2

Text Classification
Transformers
TensorBoard
Safetensors
4 languages
bert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
iati-climate-multi-classifier-weighted2 / README.md
alex-miller's picture
alex-miller
Update README.md
d1e78be verified
preview code
|
raw history blame contribute delete
No virus
4.18 kB
metadata
license: apache-2.0
base_model: alex-miller/ODABert
tags:
  - generated_from_trainer
metrics:
  - accuracy
  - f1
  - precision
  - recall
model-index:
  - name: iati-climate-multi-classifier-weighted2
    results: []
datasets:
  - alex-miller/iati-policy-markers
language:
  - en
  - fr
  - es
  - de
pipeline_tag: text-classification
widget:
  - text: >-
      VCA WWF Bolivia The programme will focus on women, young people and
      indigenous population living in the transboundary Pantanal - Chaco
      ecoregions (PACHA - Paraguay and Bolivia). Its objective is to “amplify
      their voices”, to ensure that they are participating, heard and taken into
      account in designing solutions for climate transition and common agendas
      to reach climate justice.
    example_title: Positive
  - text: >-
      HIV/AIDS prevention by education and awareness raising with emphasis on
      gender issues/El Salvador
    example_title: Negative

iati-climate-multi-classifier-weighted2

This model is a fine-tuned version of alex-miller/ODABert on a subset of the alex-miller/iati-policy-markers dataset.

It achieves the following results on the evaluation set:

  • Loss: 0.7080
  • Accuracy: 0.8541
  • F1: 0.7121
  • Precision: 0.6265
  • Recall: 0.8248

Model description

This model has been trained to identify both significant and principal climate mitigation and climate adaptation project titles and/or descriptions.

Intended uses & limitations

As many of the donors in the training dataset have mixed up Adaptation and Mitigation, the model's ability to differentiate the two isn't perfect. But the sigmoid of the model logits do bias toward the correct class.

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-06
  • train_batch_size: 24
  • eval_batch_size: 24
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20

Training results

Training Loss Epoch Step Accuracy F1 Validation Loss Precision Recall
0.7689 1.0 1951 0.7993 0.6421 0.6477 0.5264 0.8230
0.6217 2.0 3902 0.8303 0.6737 0.6269 0.5814 0.8010
0.5834 3.0 5853 0.8266 0.6761 0.6101 0.5715 0.8276
0.5571 4.0 7804 0.8461 0.6933 0.6169 0.6144 0.7954
0.5323 5.0 9755 0.8366 0.6869 0.6050 0.5913 0.8194
0.5126 6.0 11706 0.8327 0.6867 0.6047 0.5815 0.8385
0.4968 7.0 13657 0.8408 0.6938 0.6098 0.5989 0.8244
0.4893 8.0 15608 0.6040 0.8348 0.6895 0.5854 0.8387
0.4702 9.0 17559 0.6342 0.8508 0.7050 0.6211 0.8151
0.4514 10.0 19510 0.6210 0.8383 0.6946 0.5918 0.8404
0.4323 11.0 21461 0.6340 0.8402 0.6991 0.5943 0.8487
0.4193 12.0 23412 0.6407 0.8433 0.7005 0.6020 0.8375
0.407 13.0 25363 0.6602 0.8526 0.7094 0.6237 0.8223
0.3944 14.0 27314 0.6588 0.8441 0.7026 0.6029 0.8419
0.3834 15.0 29265 0.6881 0.8529 0.7110 0.6233 0.8274
0.3738 16.0 31216 0.7029 0.8575 0.7146 0.6359 0.8155
0.3686 17.0 33167 0.6929 0.8524 0.7102 0.6224 0.8271
0.3607 18.0 35118 0.7069 0.8545 0.7127 0.6272 0.8253
0.3556 19.0 37069 0.7072 0.8543 0.7118 0.6274 0.8225
0.3523 20.0 39020 0.7080 0.8541 0.7121 0.6265 0.8248

Framework versions

  • Transformers 4.41.0
  • Pytorch 2.0.1
  • Datasets 2.19.1
  • Tokenizers 0.19.1