TAPP-multilabel-bge

This model is a fine-tuned version of BAAI/bge-base-en-v1.5 on the Policy-Classification dataset.

The loss function BCEWithLogitsLoss is modified with pos_weight to focus on recall, therefore instead of loss the evaluation metrics are used to assess the model performance during training It achieves the following results on the evaluation set:

• Precision-micro: 0.7772
• Precision-samples: 0.7644
• Precision-weighted: 0.7756
• Recall-micro: 0.8329
• Recall-samples: 0.7920
• Recall-weighted: 0.8329
• F1-micro: 0.8041
• F1-samples: 0.7609
• F1-weighted: 0.8029

Model description

The purpose of this model is to predict multiple labels simultaneously from a given input data. Specifically, the model will predict four labels - ActionLabel, PlansLabel, PolicyLabel, and TargetLabel - that are relevant to a particular task or application

• Target: Targets are an intention to achieve a specific result, for example, to reduce GHG emissions to a specific level (a GHG target) or increase energy efficiency or renewable energy to a specific level (a non-GHG target), typically by
  a certain date.
• Action: Actions are an intention to implement specific means of achieving GHG reductions, usually in forms of concrete projects.
• Policies: Policies are domestic planning documents such as policies, regulations or guidlines.
• Plans:Plans are broader than specific policies or actions, such as a general intention to ‘improve efficiency’, ‘develop renewable energy’, etc.

The terms come from the World Bank's NDC platform and WRI's publication

Intended uses & limitations

More information needed

Training and evaluation data

• Training Dataset: 10031

  Class	Positive Count of Class
  Action	5416
  Plans	2140
  Policy	1396
  Target	2911

• Validation Dataset: 932

  Class	Positive Count of Class
  Action	513
  Plans	198
  Policy	122
  Target	256

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7.4e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 200
• num_epochs: 7

Training results

Environmental Impact

Carbon emissions were measured using CodeCarbon.

• Carbon Emitted: 0.07145 kg of CO2
• Hours Used: 1.36 hours

Training Hardware

• On Cloud: yes
• GPU Model: 1 x Tesla T4
• CPU Model: Intel(R) Xeon(R) CPU @ 2.30GHz
• RAM Size: 12.67 GB

Framework versions

• Transformers 4.38.1
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2