koziev ilya
commited on
Commit
•
3bb1d41
1
Parent(s):
cda61f2
notion of special tokens <s>, <sep> and </s>
Browse files
README.md
CHANGED
@@ -37,6 +37,16 @@ license: cc-by-nc-4.0
|
|
37 |
2) дисконтируем результаты п.1 символьной близостью (3-граммы) по коэффициенту Жаккара. Это штрафует перестановочные
|
38 |
перефразировки, воспроизведение исходного текста и небольшие переписывания.
|
39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
### Пример использования
|
41 |
|
42 |
Следующий код позволяет ввести в консоли короткое предложение
|
|
|
37 |
2) дисконтируем результаты п.1 символьной близостью (3-граммы) по коэффициенту Жаккара. Это штрафует перестановочные
|
38 |
перефразировки, воспроизведение исходного текста и небольшие переписывания.
|
39 |
|
40 |
+
### Формат входных данных
|
41 |
+
|
42 |
+
На вход модели подается исходный текст с добавлением токенов ```<s>``` в начале и ```<sep>``` в конце, например:
|
43 |
+
|
44 |
+
```
|
45 |
+
input_text = '<s>Мороз и солнце, день чудесный<sep>'
|
46 |
+
```
|
47 |
+
|
48 |
+
Результат генерации будет содержать текст с токеном ```</s>``` - это конец последовательности.
|
49 |
+
|
50 |
### Пример использования
|
51 |
|
52 |
Следующий код позволяет ввести в консоли короткое предложение
|