:books: add documentation
Browse files
README.md
CHANGED
@@ -50,9 +50,8 @@ card for more detailed information about the pre-training procedure.
|
|
50 |
|
51 |
## Fine-tuning
|
52 |
|
53 |
-
|
54 |
-
|
55 |
-
headers).
|
56 |
|
57 |
### Hyper parameters
|
58 |
|
|
|
50 |
|
51 |
## Fine-tuning
|
52 |
|
53 |
+
We fine-tune the model using a contrastive objective. Formally, we compute the cosine similarity from each possible sentence pairs from the batch.
|
54 |
+
We then apply the cross entropy loss by comparing with true pairs.
|
|
|
55 |
|
56 |
### Hyper parameters
|
57 |
|