File size: 7,673 Bytes
84248c3
 
ba2863f
 
 
bcb3a3f
 
84248c3
ee812da
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84248c3
beab4d1
a37b975
0c4dfaa
84248c3
0c4dfaa
84248c3
6569319
87154b0
a37b975
42d4e9f
5364210
a1321da
 
 
 
 
 
6c2f0ba
5049e56
5c06555
0ccc8b2
a1321da
5757aea
 
 
d982639
5757aea
57ae7a9
a37b975
0c4dfaa
84248c3
11c83cd
 
84248c3
11c83cd
 
84248c3
11c83cd
84248c3
11c83cd
84248c3
 
 
11c83cd
 
d833cdf
 
 
 
8ac17ff
f12450a
11c83cd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
---
language: en
tags:
- financial-sentiment-analysis
- sentiment-analysis
datasets:
- financial_phrasebank
widget:
- text: Operating profit rose to EUR 13.1 mn from EUR 8.7 mn in the corresponding
    period in 2007 representing 7.7 % of net sales.
- text: Bids or offers include at least 1,000 shares and the value of the shares must
    correspond to at least EUR 4,000.
- text: Raute reported a loss per share of EUR 0.86 for the first half of 2009 , against
    EPS of EUR 0.74 in the corresponding period of 2008.
model-index:
- name: ahmedrachid/FinancialBERT-Sentiment-Analysis
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: financial_phrasebank
      type: financial_phrasebank
      config: sentences_allagree
      split: train
    metrics:
    - type: accuracy
      value: 0.9889575971731449
      name: Accuracy
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZWMyOTZhYTA3YjdjNDkwNWVjMGRlZGQxZDM1NTBmNGFkMWM0MzM2YTJiNzI4NzBjMzFiNTMwMzVkYTJmYmNlOCIsInZlcnNpb24iOjF9.9eOX4kC5HiagnTMpBp83H8ifgjzqwSa_tzLCjH8eMxRM6EKOhd9zWIYDtPWoKvNXpODjwRYLg38xKf09p6ZxCA
    - type: f1
      value: 0.9862110528444945
      name: F1 Macro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNDBlNzhjZWU0YzIwMmIxMDkxNjk4NTkwNzA0N2RlODE5ZmNjMzVlYTBkZjJlYTlmODNiODcwMTNiZGRjYjE4NSIsInZlcnNpb24iOjF9.U_E-FCEFDIvzz7C1TWKRE0e9cSPlbV1VYy2SLAc1b-V3gonR1xUMosUwr99MTxsYSBaBAk9iyACXnefK_O45BQ
    - type: f1
      value: 0.9889575971731449
      name: F1 Micro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOGY0NTM2YThkY2VlOTZlOGZlZWMxMTU0NmIzNzNkNjIzMGI2NDM1Mjk2MzFiM2Y4MTQ5MWJmNzQxM2JmNjY1MiIsInZlcnNpb24iOjF9.6xsjHU05UtDn6vTo39MTu0Rle6CNf75dgoWqMOegs6WAW3QC6ndHhQPSGm1LriQ14IQ5J_JYK01yVXoRn1MjCg
    - type: f1
      value: 0.9889906387631547
      name: F1 Weighted
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGFmN2YwMjU1MDlkMTVjYjc5YWQ3MmQ2M2NlMWVjNWJlNDMxZjU4NTg4MjQ2NmFhZGE4OThhZjZiNjQ5N2E2OCIsInZlcnNpb24iOjF9.jvWFrjazySS_B9KZUexiATqObR826IP8eIT1O6eEZcu8GjiOCXcuNVlSfuqLFfysDWKpZXCbazSd9saUKloFCQ
    - type: precision
      value: 0.9854095875205817
      name: Precision Macro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTVhNTI0OTVhYmUxZjAxNzZkMmY4NDIwOTVlOTQ1MjA4OTZjZTNmMWZiOTg4NmFhNzY1NDViZmE3ZDFhYTZjMyIsInZlcnNpb24iOjF9.zKeviEdhTqP5Y1BmtVaBMW_3nhSd-gfXwxMVjwnaUsZNxURWUKJfCe7MACdetVtnX7Jz6ZUSybZYaZ3obUqMCw
    - type: precision
      value: 0.9889575971731449
      name: Precision Micro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZGQ3ZDViNTg0YzRlZTdlMjIyOWI5YTczNjZkZDJkNTZjNTQ5ZTc3YzY0NTI1YjMzMWQ5ODUxOWU2NzhmZjA4NSIsInZlcnNpb24iOjF9.Iaaol0A48I9ioGXYj8Tl0sWDQySxRlruUL3RiAR9NXureRbFQGuJBgF9Sd0WRrRe_0MFxkaOsXgkvBTh0u1IBg
    - type: precision
      value: 0.9891088373207723
      name: Precision Weighted
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDljNGIzYzdhOGQzYjRlNDE2ZjUwY2NjNzRhYmE1NjM4YjVkNTIwYTIyMmE4ODM5MTZkNWM1YzY2ZmRkMTc2YSIsInZlcnNpb24iOjF9.-ZULRBdW0VbSr6e64WDdKW3Ny5qT38O2lH669cQSbwp30PjPPUFO4oXhDWm4QIOjI0NfOiTjrbLTVQ7gR0vABg
    - type: recall
      value: 0.987120462774644
      name: Recall Macro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODVlYTE5NGMxMGIyM2UxN2ZhNmRiODM1ODhkZmNhNTNmMzVhNjg3M2FlYTM2NTI0MGQyN2ViM2YzODI0M2I0YyIsInZlcnNpb24iOjF9.yDZFOIzW041-s6dWxaap--K0-6Hp52hc_6rIi8_f3E-Q52WcJNLL0VHMBo0g2I3cT7UVRoIqPYoRxNgyHaZnAw
    - type: recall
      value: 0.9889575971731449
      name: Recall Micro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNmU4ZTg3MWEzNGZhMTY0MzQ1MjRmMTg1NTJmZjg0YWM3OGY4OGU5NWU0NmY0MmQ2YzZiNDYxMmFlNTNkZmUxYiIsInZlcnNpb24iOjF9.mvsikLjKldZ0SFThbAcygYEoJUNCQYE_bIbYyikMUHrSdY0BRlYsH5A32bu1BXAVMZVJVV9ebkSPmdKjZKIFAw
    - type: recall
      value: 0.9889575971731449
      name: Recall Weighted
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiMDhjNDY2Mjk0NzQyN2NjYTIxZmI5YTE1YTBkOTkzOGI3YTlmZDA1MzgxMTY4MmY3MmRkNjI4OTg4OWNmNTI0NCIsInZlcnNpb24iOjF9.zUaL-986kOJjv_VtlJAlvuEq0AxxlZaISlsmNFgvjifiFRpfPx5_-mKLkbsFjkS2q-_MQ8jTMMpQoiTVbaJMAA
    - type: loss
      value: 0.05342382565140724
      name: loss
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOTZkNzJmYWM2MzExM2YzOTUzMzJkZmIyOGNhMjNkZTU3NWRlOWEyMWE5ZGY4MDU3Yjk2MTU4NTExMTg0M2I4ZCIsInZlcnNpb24iOjF9.cwtia03w0NY4FPTj9doI3S45t50HyhjNEttRg7tcr00vA5y_6xEak7OKMXkGQZ2noribvuRyf4218STYNTHlAQ
---
### FinancialBERT for Sentiment Analysis

[*FinancialBERT*](https://huggingface.co/ahmedrachid/FinancialBERT) is a BERT model pre-trained on a large corpora of financial texts. The purpose is to enhance financial NLP research and practice in financial domain, hoping that financial practitioners and researchers can benefit from this model without the necessity of the significant computational resources required to train the model. 

The model was fine-tuned for Sentiment Analysis task on _Financial PhraseBank_ dataset. Experiments show that this model outperforms the general BERT and other financial domain-specific models.
 
More details on `FinancialBERT`'s pre-training process can be found at: https://www.researchgate.net/publication/358284785_FinancialBERT_-_A_Pretrained_Language_Model_for_Financial_Text_Mining

### Training data
FinancialBERT model was fine-tuned on [Financial PhraseBank](https://www.researchgate.net/publication/251231364_FinancialPhraseBank-v10), a dataset consisting of 4840 Financial News categorised by sentiment (negative, neutral, positive).

### Fine-tuning hyper-parameters
- learning_rate = 2e-5
- batch_size = 32
- max_seq_length = 512
- num_train_epochs = 5

### Evaluation metrics
The evaluation metrics used are: Precision, Recall and F1-score. The following is the classification report on the test set.

| sentiment  | precision        | recall           | f1-score  | support  |
| ------------- |:-------------:|:-------------:|:-------------:| -----:|
| negative | 0.96      | 0.97 | 0.97 | 58 |
| neutral | 0.98      | 0.99 | 0.98 | 279 |
| positive | 0.98     | 0.97 | 0.97 | 148 |
| macro avg | 0.97     | 0.98 | 0.98 | 485 |
| weighted avg | 0.98     | 0.98 | 0.98 | 485 |

 ### How to use 
The model can be used thanks to Transformers pipeline for sentiment analysis.
```python
from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline

model = BertForSequenceClassification.from_pretrained("ahmedrachid/FinancialBERT-Sentiment-Analysis",num_labels=3)
tokenizer = BertTokenizer.from_pretrained("ahmedrachid/FinancialBERT-Sentiment-Analysis")

nlp = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)

sentences = ["Operating profit rose to EUR 13.1 mn from EUR 8.7 mn in the corresponding period in 2007 representing 7.7 % of net sales.",  
             "Bids or offers include at least 1,000 shares and the value of the shares must correspond to at least EUR 4,000.", 
             "Raute reported a loss per share of EUR 0.86 for the first half of 2009 , against EPS of EUR 0.74 in the corresponding period of 2008.", 
             ]
results = nlp(sentences)
print(results)

[{'label': 'positive', 'score': 0.9998133778572083},
 {'label': 'neutral', 'score': 0.9997822642326355},
 {'label': 'negative', 'score': 0.9877365231513977}]
```

> Created by [Ahmed Rachid Hazourli](https://www.linkedin.com/in/ahmed-rachid/)