Update README.md
Browse files
README.md
CHANGED
|
@@ -61,7 +61,9 @@ ReMask applies an approach similar to R-TeaFor to typical chat/instruction tunin
|
|
| 61 |
Consider the following chat interaction:
|
| 62 |
|
| 63 |
> User: What is 1+1?
|
|
|
|
| 64 |
> Assistant: **1+1=2**
|
|
|
|
| 65 |
> **User:**
|
| 66 |
|
| 67 |
The model must predict the bolded parts. So, we randomly mask tokens from the bolded parts, and run the model once on the masked sequence and once on the full sequence.
|
|
|
|
| 61 |
Consider the following chat interaction:
|
| 62 |
|
| 63 |
> User: What is 1+1?
|
| 64 |
+
>
|
| 65 |
> Assistant: **1+1=2**
|
| 66 |
+
>
|
| 67 |
> **User:**
|
| 68 |
|
| 69 |
The model must predict the bolded parts. So, we randomly mask tokens from the bolded parts, and run the model once on the masked sequence and once on the full sequence.
|