:books: add documentation
Browse files
README.md
CHANGED
|
@@ -50,9 +50,8 @@ card for more detailed information about the pre-training procedure.
|
|
| 50 |
|
| 51 |
## Fine-tuning
|
| 52 |
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
headers).
|
| 56 |
|
| 57 |
### Hyper parameters
|
| 58 |
|
|
|
|
| 50 |
|
| 51 |
## Fine-tuning
|
| 52 |
|
| 53 |
+
We fine-tune the model using a contrastive objective. Formally, we compute the cosine similarity from each possible sentence pairs from the batch.
|
| 54 |
+
We then apply the cross entropy loss by comparing with true pairs.
|
|
|
|
| 55 |
|
| 56 |
### Hyper parameters
|
| 57 |
|