distilbert_ORO_screen

This model is a fine-tuned version of distilbert-base-uncased to classify whether an article's text (title + abstract + keywords) is relevant to an Ocean-related option to mitigate or adapt to climate change. Note that this model was fit on the full 'seen' dataset, therefore predictions will only be approximations of those reported in the article (in prep.).
It achieves the following results on the evaluation set:

• Train Loss: 0.3204
• Train Binary Accuracy: 0.8819
• Validation Loss: 1.5870
• Validation Binary Accuracy: 0.3519
• Epoch: 3

Model description

This model predicts a label that indicated 'relevance' for an ocean related option as a value between 0 and 1. Therefore a number > 0.5 indicates it is more likely to be relevant.

Intended uses & limitations

This model is intended to be applied to article text (title + abstract + keywords) retrieved from citation indexed databases such as Web of Science or Scopus using a search query. This can be used to autonomously screen for relevant articles from a large volume of literature and can be used in analyses that provide a granular map of the distribution of relevant studies. NB for more specific research questions, this should be complemented with manual literature review methodology (e.g. see the Collaboration for Environmental Evidence Guidelines )

Training and evaluation data

For a description of the dataset, see the paper (in prep.)

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamW', 'learning_rate': 1e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay': 0.0, 'exclude_from_weight_decay': None}
• training_precision: float32

Training results

Train Loss	Train Binary Accuracy	Validation Loss	Validation Binary Accuracy	Epoch
0.5592	0.6278	1.2920	0.2050	0
0.4488	0.7952	1.3261	0.2612	1
0.3743	0.8281	1.7808	0.2439	2
0.3204	0.8819	1.5870	0.3519	3

Framework versions

• Transformers 4.30.2
• TensorFlow 2.11.0
• Datasets 2.18.0
• Tokenizers 0.13.3