Edit model card

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Model Card for Finetuned FinBERT on Market-Based Facts

This LLM is fine-tuned on market reactions to events. By utilizing market-based data, it avoids human biases present in traditional annotation methods.

Outperform distilroberta fintuned for finance

  • +17% precision
  • +13% recall

Outperform GPT-4 zero-shot learning

  • +10% precision
  • +4% recall

Model Overview

This model is a finetuned version of FinBERT specifically tailored for the analysis of market-based facts. It was trained using headlines from leading market information providers, ensuring that the training data is both highly reliable and reflective of the information financial professionals react to. The model's objective is to classify financial headlines based on the market's return a short time after the news is released.

The finetuning process involved a distilled version of the RoBERTa-base model, following the same training methodology as DistilBERT. This distillation results in a model that is not only efficient but also retains a significant portion of the original model's predictive power. This model is case-sensitive, recognizing distinctions between uppercase and lowercase letters, which can be critical for understanding the context of financial headlines.

Model Specifications

  • Base Model: FinBERT, distilled from RoBERTa-base
  • Problem Type: Text Classification
  • Training Data: Financial headlines from leading market information providers
  • Parameters: 82 million (compared to 125M for RoBERTa-base)
  • Layers: 6
  • Dimension: 768
  • Heads: 12

Performance Metrics

The model's performance was evaluated on a validation set, yielding the following metrics:

Metric Value
Loss 0.9394
F1 Macro 0.4892
F1 Micro 0.5510
F1 Weighted 0.5189
Precision Macro 0.5241
Precision Micro 0.5510
Precision Weighted 0.5323
Recall Macro 0.5048
Recall Micro 0.5510
Recall Weighted 0.5510
Accuracy 0.5510

These metrics indicate how the model performs in terms of precision, recall, and accuracy across different averaging methods (macro, micro, and weighted).

Training and Evaluation

This model was trained using AutoTrain, which automates the training process to optimize performance on the given task. The distillation process was conducted to create a model that is both lightweight and efficient, maintaining high performance while being twice as fast as the original RoBERTa-base model.

The evaluation was conducted on a set of financial headlines not seen by the model during training, ensuring that the reported metrics reflect the model's ability to generalize to new, unseen data.

This model has been developed after publishing in the Risk Forum 2024 conference a paper that can be found here (https://arxiv.org/abs/2401.05447).

Downloads last month
0
Safetensors
Model size
82.1M params
Tensor type
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.