asahi417 commited on
Commit
807bfa5
1 Parent(s): 1114841

model update

Browse files
Files changed (1) hide show
  1. README.md +14 -14
README.md CHANGED
@@ -14,7 +14,7 @@ model-index:
14
  metrics:
15
  - name: Accuracy
16
  type: accuracy
17
- value: None
18
  - task:
19
  name: Analogy Questions (SAT full)
20
  type: multiple-choice-qa
@@ -25,7 +25,7 @@ model-index:
25
  metrics:
26
  - name: Accuracy
27
  type: accuracy
28
- value: None
29
  - task:
30
  name: Analogy Questions (SAT)
31
  type: multiple-choice-qa
@@ -36,7 +36,7 @@ model-index:
36
  metrics:
37
  - name: Accuracy
38
  type: accuracy
39
- value: None
40
  - task:
41
  name: Analogy Questions (BATS)
42
  type: multiple-choice-qa
@@ -47,7 +47,7 @@ model-index:
47
  metrics:
48
  - name: Accuracy
49
  type: accuracy
50
- value: None
51
  - task:
52
  name: Analogy Questions (Google)
53
  type: multiple-choice-qa
@@ -58,7 +58,7 @@ model-index:
58
  metrics:
59
  - name: Accuracy
60
  type: accuracy
61
- value: None
62
  - task:
63
  name: Analogy Questions (U2)
64
  type: multiple-choice-qa
@@ -69,7 +69,7 @@ model-index:
69
  metrics:
70
  - name: Accuracy
71
  type: accuracy
72
- value: None
73
  - task:
74
  name: Analogy Questions (U4)
75
  type: multiple-choice-qa
@@ -80,7 +80,7 @@ model-index:
80
  metrics:
81
  - name: Accuracy
82
  type: accuracy
83
- value: None
84
  - task:
85
  name: Lexical Relation Classification (BLESS)
86
  type: classification
@@ -160,12 +160,12 @@ RelBERT fine-tuned from [roberta-large](https://huggingface.co/roberta-large) on
160
  Fine-tuning is done via [RelBERT](https://github.com/asahi417/relbert) library (see the repository for more detail).
161
  It achieves the following results on the relation understanding tasks:
162
  - Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-nce/raw/main/analogy.json)):
163
- - Accuracy on SAT (full): None
164
- - Accuracy on SAT: None
165
- - Accuracy on BATS: None
166
- - Accuracy on U2: None
167
- - Accuracy on U4: None
168
- - Accuracy on Google: None
169
  - Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-nce/raw/main/classification.json)):
170
  - Micro F1 score on BLESS: None
171
  - Micro F1 score on CogALexV: None
@@ -173,7 +173,7 @@ It achieves the following results on the relation understanding tasks:
173
  - Micro F1 score on K&H+N: None
174
  - Micro F1 score on ROOT09: None
175
  - Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-nce/raw/main/relation_mapping.json)):
176
- - Accuracy on Relation Mapping: None
177
 
178
 
179
  ### Usage
 
14
  metrics:
15
  - name: Accuracy
16
  type: accuracy
17
+ value: 0.9688888888888889
18
  - task:
19
  name: Analogy Questions (SAT full)
20
  type: multiple-choice-qa
 
25
  metrics:
26
  - name: Accuracy
27
  type: accuracy
28
+ value: 0.6764705882352942
29
  - task:
30
  name: Analogy Questions (SAT)
31
  type: multiple-choice-qa
 
36
  metrics:
37
  - name: Accuracy
38
  type: accuracy
39
+ value: 0.6824925816023739
40
  - task:
41
  name: Analogy Questions (BATS)
42
  type: multiple-choice-qa
 
47
  metrics:
48
  - name: Accuracy
49
  type: accuracy
50
+ value: 0.783212896053363
51
  - task:
52
  name: Analogy Questions (Google)
53
  type: multiple-choice-qa
 
58
  metrics:
59
  - name: Accuracy
60
  type: accuracy
61
+ value: 0.952
62
  - task:
63
  name: Analogy Questions (U2)
64
  type: multiple-choice-qa
 
69
  metrics:
70
  - name: Accuracy
71
  type: accuracy
72
+ value: 0.6228070175438597
73
  - task:
74
  name: Analogy Questions (U4)
75
  type: multiple-choice-qa
 
80
  metrics:
81
  - name: Accuracy
82
  type: accuracy
83
+ value: 0.6481481481481481
84
  - task:
85
  name: Lexical Relation Classification (BLESS)
86
  type: classification
 
160
  Fine-tuning is done via [RelBERT](https://github.com/asahi417/relbert) library (see the repository for more detail).
161
  It achieves the following results on the relation understanding tasks:
162
  - Analogy Question ([dataset](https://huggingface.co/datasets/relbert/analogy_questions), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-nce/raw/main/analogy.json)):
163
+ - Accuracy on SAT (full): 0.6764705882352942
164
+ - Accuracy on SAT: 0.6824925816023739
165
+ - Accuracy on BATS: 0.783212896053363
166
+ - Accuracy on U2: 0.6228070175438597
167
+ - Accuracy on U4: 0.6481481481481481
168
+ - Accuracy on Google: 0.952
169
  - Lexical Relation Classification ([dataset](https://huggingface.co/datasets/relbert/lexical_relation_classification), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-nce/raw/main/classification.json)):
170
  - Micro F1 score on BLESS: None
171
  - Micro F1 score on CogALexV: None
 
173
  - Micro F1 score on K&H+N: None
174
  - Micro F1 score on ROOT09: None
175
  - Relation Mapping ([dataset](https://huggingface.co/datasets/relbert/relation_mapping), [full result](https://huggingface.co/relbert/roberta-large-semeval2012-mask-prompt-a-nce/raw/main/relation_mapping.json)):
176
+ - Accuracy on Relation Mapping: 0.9688888888888889
177
 
178
 
179
  ### Usage