snumin44 commited on
Commit
381f900
ยท
verified ยท
1 Parent(s): 2f78cfd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -3
README.md CHANGED
@@ -42,17 +42,32 @@ Multi Similarity Loss๋ฅผ ์ด์šฉํ•ด **๋™์ผํ•œ ์ฝ”๋“œ์˜ ์šฉ์–ด** ๊ฐ„์— ๋†’์€
42
  **(2) Dense Passage Retrieval (DPR)**
43
 
44
  SapBERT-KO-EN์„ ๊ฒ€์ƒ‰ ๋ชจ๋ธ๋กœ ๋งŒ๋“ค๊ธฐ ์œ„ํ•ด ์ถ”๊ฐ€์ ์ธ Fine-tuning์„ ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
45
- Bi-Encoder ๊ตฌ์กฐ๋กœ ์งˆ์˜์™€ ํ…์ŠคํŠธ์˜ ์œ ์‚ฌ๋„๋ฅผ ๊ณ„์‚ฐํ•˜๋Š” DPR ๋ฐฉ์‹์œผ๋กœ Fine-tuning์„ ์ง„ํ–‰ํ–ˆ์Šต๋‹ˆ๋‹ค.
46
 
47
  - Github : [https://github.com/snumin44/DPR-KO](https://github.com/snumin44/DPR-KO)
48
 
49
  ## 3. Training
50
 
51
- **(1) Data**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
52
 
 
53
 
54
 
55
- **(2) Training**
56
 
57
 
58
  ## 4. Example
 
42
  **(2) Dense Passage Retrieval (DPR)**
43
 
44
  SapBERT-KO-EN์„ ๊ฒ€์ƒ‰ ๋ชจ๋ธ๋กœ ๋งŒ๋“ค๊ธฐ ์œ„ํ•ด ์ถ”๊ฐ€์ ์ธ Fine-tuning์„ ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.
45
+ Bi-Encoder ๊ตฌ์กฐ๋กœ ์งˆ์˜์™€ ํ…์ŠคํŠธ์˜ ์œ ์‚ฌ๋„๋ฅผ ๊ณ„์‚ฐํ•˜๋Š” DPR ๋ฐฉ์‹์œผ๋กœ Fine-tuning ํ–ˆ์Šต๋‹ˆ๋‹ค.
46
 
47
  - Github : [https://github.com/snumin44/DPR-KO](https://github.com/snumin44/DPR-KO)
48
 
49
  ## 3. Training
50
 
51
+ **(1) Self Alignment Pretraining (SAP)**
52
+
53
+ SapBERT-KO-EN ํ•™์Šต์— ํ™œ์šฉํ•œ ๋ฒ ์ด์Šค ๋ชจ๋ธ ๋ฐ ํ•˜์ดํผ ํŒŒ๋ผ๋ฏธํ„ฐ๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์Šต๋‹ˆ๋‹ค.
54
+ ํ•œยท์˜ ์˜๋ฃŒ ์šฉ์–ด๋ฅผ ์ˆ˜๋กํ•œ ์˜๋ฃŒ ์šฉ์–ด ์‚ฌ์ „์ธ **KOSTOM**์„ ํ•™์Šต ๋ฐ์ดํ„ฐ๋กœ ์‚ฌ์šฉํ–ˆ์Šต๋‹ˆ๋‹ค.
55
+
56
+ - Model : klue/bert-base
57
+ - Dataset : **KOSTOM**
58
+ - Epochs : 1
59
+ - Batch Size : 64
60
+ - Max Length : 64
61
+ - Dropout : 0.1
62
+ - Pooler : 'cls'
63
+ - Eval Step : 100
64
+ - Threshold : 0.8
65
+ - Scale Positive Sample : 1
66
+ - Scale Negative Sample : 60
67
 
68
+ **(2) Dense Passage Retrieval (DPR)**
69
 
70
 
 
71
 
72
 
73
  ## 4. Example