---
license: apache-2.0
base_model: alex-miller/ODABert
tags:
- generated_from_trainer
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: iati-climate-multi-classifier-weighted2
  results: []
datasets:
- alex-miller/iati-policy-markers
language:
- en
- fr
- es
- de
pipeline_tag: text-classification
widget:
  - text: "VCA WWF Bolivia The programme will focus on women, young people and indigenous population living in the transboundary Pantanal - Chaco ecoregions (PACHA - Paraguay and Bolivia). Its objective is to “amplify their voices”, to ensure that they are participating, heard and taken into account in designing solutions for climate transition and common agendas to reach climate justice."
    example_title: "Positive"
  - text: "HIV/AIDS prevention by education and awareness raising with emphasis on gender issues/El Salvador"
    example_title: "Negative"
---


# iati-climate-multi-classifier-weighted2

This model is a fine-tuned version of [alex-miller/ODABert](https://huggingface.co/alex-miller/ODABert) on a subset of the [alex-miller/iati-policy-markers](https://huggingface.co/datasets/alex-miller/iati-policy-markers) dataset.

It achieves the following results on the evaluation set:
- Loss: 0.7080
- Accuracy: 0.8541
- F1: 0.7121
- Precision: 0.6265
- Recall: 0.8248

## Model description

This model has been trained to identify both significant and principal climate mitigation and climate adaptation project titles and/or descriptions.

## Intended uses & limitations

As many of the donors in the training dataset have mixed up Adaptation and Mitigation, the model's ability to differentiate the two isn't perfect. But the sigmoid of the model logits do bias toward the correct class.

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-06
- train_batch_size: 24
- eval_batch_size: 24
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 20

### Training results

| Training Loss | Epoch | Step  | Accuracy | F1     | Validation Loss | Precision | Recall |
|:-------------:|:-----:|:-----:|:--------:|:------:|:---------------:|:---------:|:------:|
| 0.7689        | 1.0   | 1951  | 0.7993   | 0.6421 | 0.6477          | 0.5264    | 0.8230 |
| 0.6217        | 2.0   | 3902  | 0.8303   | 0.6737 | 0.6269          | 0.5814    | 0.8010 |
| 0.5834        | 3.0   | 5853  | 0.8266   | 0.6761 | 0.6101          | 0.5715    | 0.8276 |
| 0.5571        | 4.0   | 7804  | 0.8461   | 0.6933 | 0.6169          | 0.6144    | 0.7954 |
| 0.5323        | 5.0   | 9755  | 0.8366   | 0.6869 | 0.6050          | 0.5913    | 0.8194 |
| 0.5126        | 6.0   | 11706 | 0.8327   | 0.6867 | 0.6047          | 0.5815    | 0.8385 |
| 0.4968        | 7.0   | 13657 | 0.8408   | 0.6938 | 0.6098          | 0.5989    | 0.8244 |
| 0.4893        | 8.0   | 15608 | 0.6040   | 0.8348 | 0.6895          | 0.5854    | 0.8387 |
| 0.4702        | 9.0   | 17559 | 0.6342   | 0.8508 | 0.7050          | 0.6211    | 0.8151 |
| 0.4514        | 10.0  | 19510 | 0.6210   | 0.8383 | 0.6946          | 0.5918    | 0.8404 |
| 0.4323        | 11.0  | 21461 | 0.6340   | 0.8402 | 0.6991          | 0.5943    | 0.8487 |
| 0.4193        | 12.0  | 23412 | 0.6407   | 0.8433 | 0.7005          | 0.6020    | 0.8375 |
| 0.407         | 13.0  | 25363 | 0.6602   | 0.8526 | 0.7094          | 0.6237    | 0.8223 |
| 0.3944        | 14.0  | 27314 | 0.6588   | 0.8441 | 0.7026          | 0.6029    | 0.8419 |
| 0.3834        | 15.0  | 29265 | 0.6881   | 0.8529 | 0.7110          | 0.6233    | 0.8274 |
| 0.3738        | 16.0  | 31216 | 0.7029   | 0.8575 | 0.7146          | 0.6359    | 0.8155 |
| 0.3686        | 17.0  | 33167 | 0.6929   | 0.8524 | 0.7102          | 0.6224    | 0.8271 |
| 0.3607        | 18.0  | 35118 | 0.7069   | 0.8545 | 0.7127          | 0.6272    | 0.8253 |
| 0.3556        | 19.0  | 37069 | 0.7072   | 0.8543 | 0.7118          | 0.6274    | 0.8225 |
| 0.3523        | 20.0  | 39020 | 0.7080   | 0.8541 | 0.7121          | 0.6265    | 0.8248 |


### Framework versions

- Transformers 4.41.0
- Pytorch 2.0.1
- Datasets 2.19.1
- Tokenizers 0.19.1