---
license: mit
tags:
- generated_from_trainer
- financial-sentiment-analysis
- sentiment-analysis
- sentence_50agree
- stocks
- sentiment
- finance
datasets:
- financial_phrasebank
- Kaggle_Self_label
- nickmuchi/financial-classification
widget:
- text: The USD rallied by 3% last night as the Fed hiked interest rates
  example_title: Bullish Sentiment
- text: >-
    Covid-19 cases have been increasing over the past few months impacting
    earnings for global firms
  example_title: Bearish Sentiment
- text: the USD has been trending lower
  example_title: Mildly Bearish Sentiment
- text: >-
    The USD rallied by 3% last night as the Fed hiked interest rates however,
    higher interest rates will increase mortgage costs for homeowners
  example_title: Neutral
metrics:
- accuracy
- f1
- precision
- recall
model-index:
- name: deberta-v3-base-finetuned-finance-text-classification
  results: []
---

# deberta-v3-base-finetuned-finance-text-classification

This model is a fine-tuned version of [microsoft/deberta-v3-base](https://huggingface.co/microsoft/deberta-v3-base) on the sentence_50Agree [financial-phrasebank + Kaggle Dataset](https://huggingface.co/datasets/nickmuchi/financial-classification), a dataset consisting of 4840 Financial News categorised by sentiment (negative, neutral, positive). The Kaggle dataset includes Covid-19 sentiment data and can be found here: [sentiment-classification-selflabel-dataset](https://www.kaggle.com/percyzheng/sentiment-classification-selflabel-dataset).

It achieves the following results on the evaluation set:
- Loss: 0.7687
- Accuracy: 0.8913
- F1: 0.8912
- Precision: 0.8927
- Recall: 0.8913

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     | Precision | Recall |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
| No log        | 1.0   | 285  | 0.4187          | 0.8399   | 0.8407 | 0.8687    | 0.8399 |
| 0.5002        | 2.0   | 570  | 0.3065          | 0.8755   | 0.8733 | 0.8781    | 0.8755 |
| 0.5002        | 3.0   | 855  | 0.4148          | 0.8775   | 0.8775 | 0.8778    | 0.8775 |
| 0.1937        | 4.0   | 1140 | 0.4249          | 0.8696   | 0.8699 | 0.8719    | 0.8696 |
| 0.1937        | 5.0   | 1425 | 0.5121          | 0.8834   | 0.8824 | 0.8831    | 0.8834 |
| 0.0917        | 6.0   | 1710 | 0.6113          | 0.8775   | 0.8779 | 0.8839    | 0.8775 |
| 0.0917        | 7.0   | 1995 | 0.7296          | 0.8775   | 0.8776 | 0.8793    | 0.8775 |
| 0.0473        | 8.0   | 2280 | 0.7034          | 0.8953   | 0.8942 | 0.8964    | 0.8953 |
| 0.0275        | 9.0   | 2565 | 0.6995          | 0.8834   | 0.8836 | 0.8846    | 0.8834 |
| 0.0275        | 10.0  | 2850 | 0.7736          | 0.8755   | 0.8755 | 0.8789    | 0.8755 |
| 0.0186        | 11.0  | 3135 | 0.7173          | 0.8814   | 0.8814 | 0.8840    | 0.8814 |
| 0.0186        | 12.0  | 3420 | 0.7659          | 0.8854   | 0.8852 | 0.8873    | 0.8854 |
| 0.0113        | 13.0  | 3705 | 0.8415          | 0.8854   | 0.8855 | 0.8907    | 0.8854 |
| 0.0113        | 14.0  | 3990 | 0.7577          | 0.8953   | 0.8951 | 0.8966    | 0.8953 |
| 0.0074        | 15.0  | 4275 | 0.7687          | 0.8913   | 0.8912 | 0.8927    | 0.8913 |


### Framework versions

- Transformers 4.19.2
- Pytorch 1.11.0+cu113
- Datasets 2.2.2
- Tokenizers 0.12.1