Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,37 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
+
language:
|
4 |
+
- ko
|
5 |
---
|
6 |
+
# klue-cross-encoder-v1
|
7 |
+
- klue/bert-base ๋ชจ๋ธ์ ํ๋ จ์์ผ cross-encoder๋ก ํ์ธํ๋ํ ๋ชจ๋ธ
|
8 |
+
- This model was trained using [SentenceTransformers](https://sbert.net) [Cross-Encoder](https://www.sbert.net/examples/applications/cross-encoder/README.html) class.
|
9 |
+
|
10 |
+
# Training
|
11 |
+
- sts(10)-sts(10)ํ๋ จ ์ํด
|
12 |
+
- STS : seed=111,epoch=10, lr=1e-4, eps=1e-6, warm_step=10%, max_seq_len=128, train_batch=128(small ๋ชจ๋ธ=32) (albert 13m/7G) [ํ๋ จ์ฝ๋](https://github.com/kobongsoo/BERT/blob/master/sbert/cross-encoder/sbert-corossencoder-train-nli.ipynb)
|
13 |
+
- [ํ๊ฐ์ฝ๋](https://github.com/kobongsoo/BERT/blob/master/sbert/cross-encoder/sbert-crossencoder-test3.ipynb),[ํ
์คํธ์ฝ๋](https://github.com/kobongsoo/BERT/blob/master/sbert/cross-encoder/sbert-crossencoder-test.ipynb)
|
14 |
+
|
15 |
+
-
|
16 |
+
|๋ชจ๋ธ |korsts|klue-sts|glue(stsb)|stsb_multi_mt(en)|
|
17 |
+
|:--------|------:|--------:|--------------:|------------:|
|
18 |
+
|albert-small-kor-cross-encoder-v1 |0.8455 |0.8526 |0.8513 |0.7976|
|
19 |
+
|**klue-cross-encoder-v1** |0.8262 |0.8833 |0.8512 |0.7889|
|
20 |
+
|kpf-cross-encoder-v1 |0.8799 |0.9133 |0.8626 |0.8027|
|
21 |
+
|
22 |
+
## Usage and Performance
|
23 |
+
|
24 |
+
Pre-trained models can be used like this:
|
25 |
+
```
|
26 |
+
from sentence_transformers import CrossEncoder
|
27 |
+
model = CrossEncoder('bongsoo/kpf-cross-encoder-v1')
|
28 |
+
scores = model.predict([('์ค๋ ๋ ์จ๊ฐ ์ข๋ค', '์ค๋ ๋ฑ์ฐ์ ํ๋ค'), ('์ค๋ ๋ ์จ๊ฐ ํ๋ฆฌ๋ค', '์ค๋ ๋น๊ฐ ๋ด๋ฆฐ๋ค')])
|
29 |
+
print(scores)
|
30 |
+
```
|
31 |
+
```
|
32 |
+
[0.10161418 0.45563662]
|
33 |
+
```
|
34 |
+
|
35 |
+
The model will predict scores for the pairs `('Sentence 1', 'Sentence 2')` and `('Sentence 3', 'Sentence 4')`.
|
36 |
+
|
37 |
+
You can use this model also without sentence_transformers and by just using Transformers ``AutoModel`` class
|