asahi417 commited on
Commit
86e7bf9
1 Parent(s): f8541f8

model update

Browse files
Files changed (1) hide show
  1. README.md +29 -29
README.md CHANGED
@@ -14,7 +14,7 @@ model-index:
14
  metrics:
15
  - name: Accuracy
16
  type: accuracy
17
- value: None
18
  - task:
19
  name: Analogy Questions (SAT full)
20
  type: multiple-choice-qa
@@ -25,7 +25,7 @@ model-index:
25
  metrics:
26
  - name: Accuracy
27
  type: accuracy
28
- value: None
29
  - task:
30
  name: Analogy Questions (SAT)
31
  type: multiple-choice-qa
@@ -36,7 +36,7 @@ model-index:
36
  metrics:
37
  - name: Accuracy
38
  type: accuracy
39
- value: None
40
  - task:
41
  name: Analogy Questions (BATS)
42
  type: multiple-choice-qa
@@ -47,7 +47,7 @@ model-index:
47
  metrics:
48
  - name: Accuracy
49
  type: accuracy
50
- value: None
51
  - task:
52
  name: Analogy Questions (Google)
53
  type: multiple-choice-qa
@@ -58,7 +58,7 @@ model-index:
58
  metrics:
59
  - name: Accuracy
60
  type: accuracy
61
- value: None
62
  - task:
63
  name: Analogy Questions (U2)
64
  type: multiple-choice-qa
@@ -69,7 +69,7 @@ model-index:
69
  metrics:
70
  - name: Accuracy
71
  type: accuracy
72
- value: None
73
  - task:
74
  name: Analogy Questions (U4)
75
  type: multiple-choice-qa
@@ -80,7 +80,7 @@ model-index:
80
  metrics:
81
  - name: Accuracy
82
  type: accuracy
83
- value: None
84
  - task:
85
  name: Lexical Relation Classification (BLESS)
86
  type: classification
@@ -91,10 +91,10 @@ model-index:
91
  metrics:
92
  - name: F1
93
  type: f1
94
- value: None
95
  - name: F1 (macro)
96
  type: f1_macro
97
- value: None
98
  - task:
99
  name: Lexical Relation Classification (CogALexV)
100
  type: classification
@@ -105,10 +105,10 @@ model-index:
105
  metrics:
106
  - name: F1
107
  type: f1
108
- value: None
109
  - name: F1 (macro)
110
  type: f1_macro
111
- value: None
112
  - task:
113
  name: Lexical Relation Classification (EVALution)
114
  type: classification
@@ -119,10 +119,10 @@ model-index:
119
  metrics:
120
  - name: F1
121
  type: f1
122
- value: None
123
  - name: F1 (macro)
124
  type: f1_macro
125
- value: None
126
  - task:
127
  name: Lexical Relation Classification (K&H+N)
128
  type: classification
@@ -133,10 +133,10 @@ model-index:
133
  metrics:
134
  - name: F1
135
  type: f1
136
- value: None
137
  - name: F1 (macro)
138
  type: f1_macro
139
- value: None
140
  - task:
141
  name: Lexical Relation Classification (ROOT09)
142
  type: classification
@@ -147,10 +147,10 @@ model-index:
147
  metrics:
148
  - name: F1
149
  type: f1
150
- value: None
151
  - name: F1 (macro)
152
  type: f1_macro
153
- value: None
154
 
155
  ---
156
  # relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated
@@ -160,20 +160,20 @@ RelBERT fine-tuned from [roberta-large](https://huggingface.co/roberta-large) on
160
  Fine-tuning is done via [RelBERT](https://github.com/asahi417/relbert) library (see the repository for more detail).
161
  It achieves the following results on the relation understanding tasks:
162
  - Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated/raw/main/analogy.json)):
163
- - Accuracy on SAT (full): None
164
- - Accuracy on SAT: None
165
- - Accuracy on BATS: None
166
- - Accuracy on U2: None
167
- - Accuracy on U4: None
168
- - Accuracy on Google: None
169
  - Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated/raw/main/classification.json)):
170
- - Micro F1 score on BLESS: None
171
- - Micro F1 score on CogALexV: None
172
- - Micro F1 score on EVALution: None
173
- - Micro F1 score on K&H+N: None
174
- - Micro F1 score on ROOT09: None
175
  - Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated/raw/main/relation_mapping.json)):
176
- - Accuracy on Relation Mapping: None
177
 
178
 
179
  ### Usage
 
14
  metrics:
15
  - name: Accuracy
16
  type: accuracy
17
+ value: 0.8453174603174604
18
  - task:
19
  name: Analogy Questions (SAT full)
20
  type: multiple-choice-qa
 
25
  metrics:
26
  - name: Accuracy
27
  type: accuracy
28
+ value: 0.6310160427807486
29
  - task:
30
  name: Analogy Questions (SAT)
31
  type: multiple-choice-qa
 
36
  metrics:
37
  - name: Accuracy
38
  type: accuracy
39
+ value: 0.6379821958456974
40
  - task:
41
  name: Analogy Questions (BATS)
42
  type: multiple-choice-qa
 
47
  metrics:
48
  - name: Accuracy
49
  type: accuracy
50
+ value: 0.7581989994441356
51
  - task:
52
  name: Analogy Questions (Google)
53
  type: multiple-choice-qa
 
58
  metrics:
59
  - name: Accuracy
60
  type: accuracy
61
+ value: 0.912
62
  - task:
63
  name: Analogy Questions (U2)
64
  type: multiple-choice-qa
 
69
  metrics:
70
  - name: Accuracy
71
  type: accuracy
72
+ value: 0.5131578947368421
73
  - task:
74
  name: Analogy Questions (U4)
75
  type: multiple-choice-qa
 
80
  metrics:
81
  - name: Accuracy
82
  type: accuracy
83
+ value: 0.6087962962962963
84
  - task:
85
  name: Lexical Relation Classification (BLESS)
86
  type: classification
 
91
  metrics:
92
  - name: F1
93
  type: f1
94
+ value: 0.9308422480036161
95
  - name: F1 (macro)
96
  type: f1_macro
97
+ value: 0.9259132485159167
98
  - task:
99
  name: Lexical Relation Classification (CogALexV)
100
  type: classification
 
105
  metrics:
106
  - name: F1
107
  type: f1
108
+ value: 0.8812206572769953
109
  - name: F1 (macro)
110
  type: f1_macro
111
+ value: 0.7406199625428094
112
  - task:
113
  name: Lexical Relation Classification (EVALution)
114
  type: classification
 
119
  metrics:
120
  - name: F1
121
  type: f1
122
+ value: 0.7231852654387866
123
  - name: F1 (macro)
124
  type: f1_macro
125
+ value: 0.7092518484577178
126
  - task:
127
  name: Lexical Relation Classification (K&H+N)
128
  type: classification
 
133
  metrics:
134
  - name: F1
135
  type: f1
136
+ value: 0.9675871183139737
137
  - name: F1 (macro)
138
  type: f1_macro
139
+ value: 0.9013308490394988
140
  - task:
141
  name: Lexical Relation Classification (ROOT09)
142
  type: classification
 
147
  metrics:
148
  - name: F1
149
  type: f1
150
+ value: 0.9160137887809464
151
  - name: F1 (macro)
152
  type: f1_macro
153
+ value: 0.9146918325555718
154
 
155
  ---
156
  # relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated
 
160
  Fine-tuning is done via [RelBERT](https://github.com/asahi417/relbert) library (see the repository for more detail).
161
  It achieves the following results on the relation understanding tasks:
162
  - Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated/raw/main/analogy.json)):
163
+ - Accuracy on SAT (full): 0.6310160427807486
164
+ - Accuracy on SAT: 0.6379821958456974
165
+ - Accuracy on BATS: 0.7581989994441356
166
+ - Accuracy on U2: 0.5131578947368421
167
+ - Accuracy on U4: 0.6087962962962963
168
+ - Accuracy on Google: 0.912
169
  - Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated/raw/main/classification.json)):
170
+ - Micro F1 score on BLESS: 0.9308422480036161
171
+ - Micro F1 score on CogALexV: 0.8812206572769953
172
+ - Micro F1 score on EVALution: 0.7231852654387866
173
+ - Micro F1 score on K&H+N: 0.9675871183139737
174
+ - Micro F1 score on ROOT09: 0.9160137887809464
175
  - Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-c-loob-conceptnet-validated/raw/main/relation_mapping.json)):
176
+ - Accuracy on Relation Mapping: 0.8453174603174604
177
 
178
 
179
  ### Usage