koziev ilya commited on
Commit
3bb1d41
1 Parent(s): cda61f2

notion of special tokens <s>, <sep> and </s>

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -37,6 +37,16 @@ license: cc-by-nc-4.0
37
  2) дисконтируем результаты п.1 символьной близостью (3-граммы) по коэффициенту Жаккара. Это штрафует перестановочные
38
  перефразировки, воспроизведение исходного текста и небольшие переписывания.
39
 
 
 
 
 
 
 
 
 
 
 
40
  ### Пример использования
41
 
42
  Следующий код позволяет ввести в консоли короткое предложение
 
37
  2) дисконтируем результаты п.1 символьной близостью (3-граммы) по коэффициенту Жаккара. Это штрафует перестановочные
38
  перефразировки, воспроизведение исходного текста и небольшие переписывания.
39
 
40
+ ### Формат входных данных
41
+
42
+ На вход модели подается исходный текст с добавлением токенов ```<s>``` в начале и ```<sep>``` в конце, например:
43
+
44
+ ```
45
+ input_text = '<s>Мороз и солнце, день чудесный<sep>'
46
+ ```
47
+
48
+ Результат генерации будет содержать текст с токеном ```</s>``` - это конец последовательности.
49
+
50
  ### Пример использования
51
 
52
  Следующий код позволяет ввести в консоли короткое предложение