---
license: apache-2.0
tags:
- generated_from_keras_callback
model-index:
- name: distilbert_ORO_Branch
  results: []
widget:
- text: Generation of energy across the world is today reliant majorly on fossil fuels. The burning of these fuels is growing in line with the increase in the demand for energy globally. Consequently, climate change, air contamination, and energy security issues are rising as well. An efficient alternative to this grave hazard is the speedy substitution of fossil fuel-based carbon energy sources with the shift to clean sources of renewable energy that cause zero emissions. This needs to happen in conjunction with the continuing increase in the overall consumption of energy worldwide. Many resources of renewable energy are available. These include thermal, solar photovoltaic, biomass and wind, tidal energy, hydropower, and geothermal. Notably, tidal energy exhibits great potential with regard to its dependability, superior energy density, certainty, and durability. The energy mined from the tides on the basis of steady and anticipated vertical movements of the water, causing tidal currents, could be converted into kinetic energy to produce electricity. Tidal barrages could channel mechanical energy, while tidewater river turbines can seize the energy from tidal currents. This study discusses the present trends, ecological effects, and the prospects for technology related to tidal energy.
---

# distilbert_ORO_Branch

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) to classify an article's text (title + abstract + keywords). The intention is that this model will be used AFTER establishing an article's relevance to an Ocean-related option (ORO) (see the [screening model card on huggingface](https://huggingface.co/dveytia/distilbert_ORO_screen)). This model will then classify a relevant article futher into the type of ORO: Mitigation, Natural resilence or Societal adaptation.

It achieves the following results on the evaluation set:
- Train Loss: 0.1604
- Train Binary Accuracy: 0.9509
- Validation Loss: 0.2817
- Validation Binary Accuracy: 0.8946
- Epoch: 2

## Model description

This model predicts for relevance to three labels specifying the type of ocean related option as a value between 0 and 1. Therefore a number > 0.5 indicates it is more likely to be relevant that that type of ORO.

## Intended uses & limitations

This model is intended to be applied to article text (title + abstract + keywords) retrieved from citation indexed databases such as Web of Science or Scopus using a search query. This can be used to autonomously classify relevant articles from a large volume of literature and can be used in analyses that provide a granular map of the distribution of relevant studies. 

## Training and evaluation data

For a description of the dataset, see the paper (in prep.)

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- optimizer: {'name': 'AdamW', 'learning_rate': 1e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay': 0.0, 'exclude_from_weight_decay': None}
- training_precision: float32

### Training results

| Train Loss | Train Binary Accuracy | Validation Loss | Validation Binary Accuracy | Epoch |
|:----------:|:---------------------:|:---------------:|:--------------------------:|:-----:|
| 0.4863     | 0.7650                | 0.4002          | 0.8368                     | 0     |
| 0.2445     | 0.9238                | 0.2649          | 0.8993                     | 1     |
| 0.1604     | 0.9509                | 0.2817          | 0.8946                     | 2     |


### Framework versions

- Transformers 4.30.2
- TensorFlow 2.12.0
- Datasets 2.18.0
- Tokenizers 0.13.3