Hugging Face's logo

nlpaueb
/
sec-bert-base

Fill-Mask
Transformers
PyTorch
TensorFlow
English
bert
pretraining
finance
financial
Inference Endpoints
Model card Files Community
2
nlpaueb commited on
Commit
4405a3a
•
1 Parent(s): 6cc4c65

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +121 -41
README.md CHANGED
@@ -58,49 +58,129 @@ model = AutoModel.from_pretrained("nlpaueb/sec-bert-base")
58
  | --------------------------------------------------- | ------------ |
59
  | **BERT-BASE-UNCASED** | increased (0.221), were (0.131), are (0.103), rose (0.075), of (0.058)
60
  | **SEC-BERT-BASE** | increased (0.678), decreased (0.282), declined (0.017), grew (0.016), rose (0.004)
61
- | **SEC-BERT-NUM** | increased (0.665), decreased (0.281), grew (0.028), declined (0.015), rose (0.008)
62
- | **SEC-BERT-SHAPE** | increased (0.793), decreased (0.145), grew (0.042), declined (0.011), rose (0.003)
63
-
64
- | Model | Masked Token | Predictions (Probability) |
65
- | ---------------------------------- | ------------ | ------------ |
66
- | Total net sales [MASK] 2% or $5.4 billion during 2019 compared to 2018.
67
- | **BERT-BASE-UNCASED** | decreased | increased (0.221), were (0.131), are (0.103), rose (0.075), of (0.058)
68
- | **SEC-BERT-BASE** | decreased | increased (0.678), decreased (0.282), declined (0.017), grew (0.016), rose (0.004)
69
- | **SEC-BERT-NUM** | decreased | increased (0.665), decreased (0.281), grew (0.028), declined (0.015), rose (0.008)
70
- | **SEC-BERT-SHAPE** | decreased | increased (0.793), decreased (0.145), grew (0.042), declined (0.011), rose (0.003)
71
- | | |
72
- | Total net sales decreased 2% or $5.4 [MASK] during 2019 compared to 2018.
73
- | **BERT-BASE-UNCASED** | billion |
74
- | **SEC-BERT-BASE** | billion |
75
- | **SEC-BERT-NUM** | billion |
76
- | **SEC-BERT-SHAPE** | billion |
77
- | | |
78
- | During 2019, the Company [MASK] $67.1 billion of its common stock and paid dividend equivalents of $14.1 billion.
79
- | **BERT-BASE-UNCASED** | repurchased | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
80
- | **SEC-BERT-BASE** | repurchased | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
81
- | **SEC-BERT-NUM** | repurchased | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
82
- | **SEC-BERT-SHAPE** | repurchased | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
83
- | | |
84
- | During 2019, the Company repurchased $67.1 billion of its common [MASK] and paid dividend equivalents of $14.1 billion.
85
- | **BERT-BASE-UNCASED** | stock | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
86
- | **SEC-BERT-BASE** | stock | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
87
- | **SEC-BERT-NUM** | stock | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
88
- | **SEC-BERT-SHAPE** | stock | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
89
- | | |
90
- | During 2019, the Company repurchased $67.1 billion of its common stock and paid [MASK] equivalents of $14.1 billion.
91
- | **BERT-BASE-UNCASED** | dividend | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
92
- | **SEC-BERT-BASE** | dividend | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
93
- | **SEC-BERT-NUM** | dividend | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
94
- | **SEC-BERT-SHAPE** | dividend | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
95
- | | |
96
- | During 2019, the Company repurchased $67.1 billion of its common stock and paid dividend [MASK] of $14.1 billion.
97
- | **BERT-BASE-UNCASED** | equivalents | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
98
- | **SEC-BERT-BASE** | equivalents | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
99
- | **SEC-BERT-NUM** | equivalents | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
100
- | **SEC-BERT-SHAPE** | equivalents | ('letter', '0.38'), ('dealer', '0.04'), ('employment', '0.03'), ('award', '0.03'), ('contribution', '0.02')
101
- | | |
102
 
103
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
104
  ## About Us
105
 
106
  [AUEB's Natural Language Processing Group](http://nlp.cs.aueb.gr) develops algorithms, models, and systems that allow computers to process and generate natural language texts.
 
58
  | --------------------------------------------------- | ------------ |
59
  | **BERT-BASE-UNCASED** | increased (0.221), were (0.131), are (0.103), rose (0.075), of (0.058)
60
  | **SEC-BERT-BASE** | increased (0.678), decreased (0.282), declined (0.017), grew (0.016), rose (0.004)
61
+ | **SEC-BERT-NUM** | increased (0.753), decreased (0.211), grew (0.019), declined (0.010), rose (0.006)
62
+ | **SEC-BERT-SHAPE** | increased (0.747), decreased (0.214), grew (0.021), declined (0.013), rose (0.002)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
 
64
 
65
+ | Sample | Masked Token |
66
+ | --------------------------------------------------- | ------------ |
67
+ | Total net sales decreased 2% or $5.4 [MASK] during 2019 compared to 2018. | billion
68
+
69
+ | Model | Predictions (Probability) |
70
+ | --------------------------------------------------- | ------------ |
71
+ | **BERT-BASE-UNCASED** | billion (0.841), million (0.097), trillion (0.028), ##m (0.015), ##bn (0.006)
72
+ | **SEC-BERT-BASE** | million (0.972), billion (0.028), millions (0.000), ##million (0.000), m (0.000)
73
+ | **SEC-BERT-NUM** | million (0.974), billion (0.012), , (0.010), thousand (0.003), m (0.000)
74
+ | **SEC-BERT-SHAPE** | million (0.978), billion (0.021), % (0.000), , (0.000), millions (0.000)
75
+
76
+
77
+ | Sample | Masked Token |
78
+ | --------------------------------------------------- | ------------ |
79
+ | Total net sales decreased [MASK]% or $5.4 billion during 2019 compared to 2018. | 2
80
+
81
+ | Model | Predictions (Probability) |
82
+ | --------------------------------------------------- | ------------ |
83
+ | **BERT-BASE-UNCASED** | 20 (0.031), 10 (0.030), 6 (0.029), 4 (0.027), 30 (0.027)
84
+ | **SEC-BERT-BASE** | 13 (0.045), 12 (0.040), 11 (0.040), 14 (0.035), 10 (0.035)
85
+ | **SEC-BERT-NUM** | [NUM] (1.000), one (0.000), five (0.000), three (0.000), seven (0.000)
86
+ | **SEC-BERT-SHAPE** | [XX] (0.316), [XX.X] (0.253), [X.X] (0.237), [X] (0.188), [X.XX] (0.002)
87
+
88
+
89
+ | Sample | Masked Token |
90
+ | --------------------------------------------------- | ------------ |
91
+ | Total net sales decreased 2[MASK] or $5.4 billion during 2019 compared to 2018. | %
92
+
93
+ | Model | Predictions (Probability) |
94
+ | --------------------------------------------------- | ------------ |
95
+ | **BERT-BASE-UNCASED** | % (0.795), percent (0.174), ##fold (0.009), billion (0.004), times (0.004)
96
+ | **SEC-BERT-BASE** | % (0.924), percent (0.076), points (0.000), , (0.000), times (0.000)
97
+ | **SEC-BERT-NUM** | % (0.882), percent (0.118), million (0.000), units (0.000), bps (0.000)
98
+ | **SEC-BERT-SHAPE** | % (0.961), percent (0.039), bps (0.000), , (0.000), bcf (0.000)
99
+
100
+
101
+ | Sample | Masked Token |
102
+ | --------------------------------------------------- | ------------ |
103
+ | Total net sales decreased 2% or $[MASK] billion during 2019 compared to 2018. | 5.4
104
+
105
+ | Model | Predictions (Probability) |
106
+ | --------------------------------------------------- | ------------ |
107
+ | **BERT-BASE-UNCASED** | 1 (0.074), 4 (0.045), 3 (0.044), 2 (0.037), 5 (0.034)
108
+ | **SEC-BERT-BASE** | 1 (0.218), 2 (0.136), 3 (0.078), 4 (0.066), 5 (0.048)
109
+ | **SEC-BERT-NUM** | [NUM] (1.000), l (0.000), 1 (0.000), - (0.000), 30 (0.000)
110
+ | **SEC-BERT-SHAPE** | [X.X] (0.787), [X.XX] (0.095), [XX.X] (0.049), [X.XXX] (0.046), [X] (0.013)
111
+
112
+
113
+ | Sample | Masked Token |
114
+ | --------------------------------------------------- | ------------ |
115
+ | Total net sales decreased 2% or $5.4 billion during [MASK] compared to 2018. | 2019
116
+
117
+ | Model | Predictions (Probability) |
118
+ | --------------------------------------------------- | ------------ |
119
+ | **BERT-BASE-UNCASED** | 2017 (0.485), 2018 (0.169), 2016 (0.164), 2015 (0.070), 2014 (0.022)
120
+ | **SEC-BERT-BASE** | 2019 (0.990), 2017 (0.007), 2018 (0.003), 2020 (0.000), 2015 (0.000)
121
+ | **SEC-BERT-NUM** | [NUM] (1.000), as (0.000), fiscal (0.000), year (0.000), when (0.000)
122
+ | **SEC-BERT-SHAPE** | [XXXX] (1.000), as (0.000), year (0.000), periods (0.000), , (0.000)
123
+
124
+
125
+ | Sample | Masked Token |
126
+ | --------------------------------------------------- | ------------ |
127
+ | Total net sales decreased 2% or $5.4 billion during 2019 compared to [MASK]. | 2018
128
+
129
+ | Model | Predictions (Probability) |
130
+ | --------------------------------------------------- | ------------ |
131
+ | **BERT-BASE-UNCASED** | 2017 (0.100), 2016 (0.097), above (0.054), inflation (0.050), previously (0.037)
132
+ | **SEC-BERT-BASE** | 2018 (0.999), 2019 (0.000), 2017 (0.000), 2016 (0.000), 2014 (0.000)
133
+ | **SEC-BERT-NUM** | [NUM] (1.000), year (0.000), last (0.000), sales (0.000), fiscal (0.000)
134
+ | **SEC-BERT-SHAPE** | [XXXX] (1.000), year (0.000), sales (0.000), prior (0.000), years (0.000)
135
+
136
+
137
+ | Sample | Masked Token |
138
+ | --------------------------------------------------- | ------------ |
139
+ | During 2019, the Company [MASK] $67.1 billion of its common stock and paid dividend equivalents of $14.1 billion. | repurchased
140
+
141
+ | Model | Predictions (Probability) |
142
+ | --------------------------------------------------- | ------------ |
143
+ | **BERT-BASE-UNCASED** | held (0.229), sold (0.192), acquired (0.172), owned (0.052), traded (0.033)
144
+ | **SEC-BERT-BASE** | repurchased (0.913), issued (0.036), purchased (0.029), redeemed (0.010), sold (0.003)
145
+ | **SEC-BERT-NUM** | repurchased (0.917), purchased (0.054), reacquired (0.013), issued (0.005), acquired (0.003)
146
+ | **SEC-BERT-SHAPE** | repurchased (0.902), purchased (0.068), issued (0.010), reacquired (0.008), redeemed (0.006)
147
+
148
+
149
+ | Sample | Masked Token |
150
+ | --------------------------------------------------- | ------------ |
151
+ | During 2019, the Company repurchased $67.1 billion of its common [MASK] and paid dividend equivalents of $14.1 billion. | stock
152
+
153
+ | Model | Predictions (Probability) |
154
+ | --------------------------------------------------- | ------------ |
155
+ | **BERT-BASE-UNCASED** | stock (0.835), assets (0.039), equity (0.025), debt (0.021), bonds (0.017)
156
+ | **SEC-BERT-BASE** | stock (0.857), shares (0.135), equity (0.004), units (0.002), securities (0.000)
157
+ | **SEC-BERT-NUM** | stock (0.842), shares (0.157), equity (0.000), securities (0.000), units (0.000)
158
+ | **SEC-BERT-SHAPE** | stock (0.888), shares (0.109), equity (0.001), securities (0.001), stocks (0.000)
159
+
160
+
161
+ | Sample | Masked Token |
162
+ | --------------------------------------------------- | ------------ |
163
+ | During 2019, the Company repurchased $67.1 billion of its common stock and paid [MASK] equivalents of $14.1 billion. | dividend
164
+
165
+ | Model | Predictions (Probability) |
166
+ | --------------------------------------------------- | ------------ |
167
+ | **BERT-BASE-UNCASED** | cash (0.276), net (0.128), annual (0.083), the (0.040), debt (0.027)
168
+ | **SEC-BERT-BASE** | dividend (0.890), cash (0.018), dividends (0.016), share (0.013), tax (0.010)
169
+ | **SEC-BERT-NUM** | dividend (0.735), cash (0.115), share (0.087), tax (0.025), stock (0.013)
170
+ | **SEC-BERT-SHAPE** | dividend (0.655), cash (0.248), dividends (0.042), share (0.019), out (0.003)
171
+
172
+
173
+ | Sample | Masked Token |
174
+ | --------------------------------------------------- | ------------ |
175
+ | During 2019, the Company repurchased $67.1 billion of its common stock and paid dividend [MASK] of $14.1 billion. | equivalents
176
+
177
+ | Model | Predictions (Probability) |
178
+ | --------------------------------------------------- | ------------ |
179
+ | **BERT-BASE-UNCASED** | revenue (0.085), earnings (0.078), rates (0.065), amounts (0.064), proceeds (0.062)
180
+ | **SEC-BERT-BASE** | payments (0.790), distributions (0.087), equivalents (0.068), cash (0.013), amounts (0.004)
181
+ | **SEC-BERT-NUM** | payments (0.845), equivalents (0.097), distributions (0.024), increases (0.005), dividends (0.004)
182
+ | **SEC-BERT-SHAPE** | payments (0.784), equivalents (0.093), distributions (0.043), dividends (0.015), requirements (0.009)
183
+
184
  ## About Us
185
 
186
  [AUEB's Natural Language Processing Group](http://nlp.cs.aueb.gr) develops algorithms, models, and systems that allow computers to process and generate natural language texts.