Update README.md
Browse files
README.md
CHANGED
|
@@ -1,5 +1,4 @@
|
|
| 1 |
---
|
| 2 |
-
pipeline_tag: sentence-similarity
|
| 3 |
tags:
|
| 4 |
- sentence-transformers
|
| 5 |
- feature-extraction
|
|
@@ -10,10 +9,31 @@ datasets:
|
|
| 10 |
license: cc-by-sa-4.0
|
| 11 |
language:
|
| 12 |
- ja
|
|
|
|
|
|
|
|
|
|
| 13 |
---
|
| 14 |
|
| 15 |
# sup-simcse-ja-base
|
| 16 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
## Usage (Sentence-Transformers)
|
| 19 |
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
tags:
|
| 3 |
- sentence-transformers
|
| 4 |
- feature-extraction
|
|
|
|
| 9 |
license: cc-by-sa-4.0
|
| 10 |
language:
|
| 11 |
- ja
|
| 12 |
+
metrics:
|
| 13 |
+
- spearmanr
|
| 14 |
+
library_name: sentence-transformers
|
| 15 |
---
|
| 16 |
|
| 17 |
# sup-simcse-ja-base
|
| 18 |
|
| 19 |
+
## Model Summary
|
| 20 |
+
|
| 21 |
+
- Fine-tuning method: Supervised SimCSE
|
| 22 |
+
- Base model: [cl-tohoku/bert-base-japanese-v3](https://huggingface.co/cl-tohoku/bert-base-japanese-v3)
|
| 23 |
+
- Training dataset: [JSNLI](https://nlp.ist.i.kyoto-u.ac.jp/?%E6%97%A5%E6%9C%AC%E8%AA%9ESNLI%28JSNLI%29%E3%83%87%E3%83%BC%E3%82%BF%E3%82%BB%E3%83%83%E3%83%88)
|
| 24 |
+
- Pooling strategy: cls (with an extra MLP layer only during training)
|
| 25 |
+
- Hidden size: 768
|
| 26 |
+
- Learning rate: 5e-5
|
| 27 |
+
- Batch size: 512
|
| 28 |
+
- Temperature: 0.05
|
| 29 |
+
- Max sequence length: 64
|
| 30 |
+
- Number of training examples: 2^20
|
| 31 |
+
- Validation interval (steps): 2^6
|
| 32 |
+
- Warmup ratio: 0.1
|
| 33 |
+
- Dtype: BFloat16
|
| 34 |
+
|
| 35 |
+
See the [GitHub repository](https://github.com/hppRC/simple-simcse-ja) for a detailed experimental setup.
|
| 36 |
+
|
| 37 |
|
| 38 |
## Usage (Sentence-Transformers)
|
| 39 |
|