title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° ÑÐ¸ÑÑÐµÐ¼Ñ Ð¸Ð·Ð²Ð»ÐµÑÐµÐ½Ð¸Ñ Ð¾Ð¿Ð¸ÑÐ°Ð½Ð¸Ñ ÑÑÑÑÐºÑÑÑÑ Ð´Ð¸ÑÐºÑÑÑÐ° Ð¸Ð· ÑÐµÐºÑÑÐ¾Ð² Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð²
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Development of a system for extracting a description of the structure of discourse from dialogue texts
Creators: name: ÐÐ°Ð½ÑÐµÐ²Ð°, ÐÐ½Ð°ÑÑÐ°ÑÐ¸Ñ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2020
Subjects: Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ° ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°
ÑÑÑÑÐºÑÑÑÐ° Ð´Ð¸ÑÐºÑÑÑÐ°
Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¸ ÑÐ¾ Ð¼Ð½Ð¾Ð³Ð¸Ð¼Ð¸ ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ°Ð¼Ð¸
stac
natural language processing
discourse structure
multi-party dialogues

description: Ð¢ÐµÐ¼Ð° Ð²ÑÐ¿ÑÑÐºÐ½Ð¾Ð¹ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ: Â«Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° ÑÐ¸ÑÑÐµÐ¼Ñ Ð¸Ð·Ð²Ð»ÐµÑÐµÐ½Ð¸Ñ Ð¾Ð¿Ð¸ÑÐ°Ð½Ð¸Ñ ÑÑÑÑÐºÑÑÑÑ Ð´Ð¸ÑÐºÑÑÑÐ° Ð¸Ð· ÑÐµÐºÑÑÐ¾Ð² Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð²Â».
ÐÐ°Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° Ð²Ð¾ÑÑÑÐ°Ð½Ð¾Ð²Ð»ÐµÐ½Ð¸Ñ ÑÑÑÑÐºÑÑÑÑ Ð´Ð¸ÑÐºÑÑÑÐ° Ð² Ð´Ð¸Ð°Ð»Ð¾Ð³Ð°Ñ Ñ Ð¼Ð½Ð¾Ð³Ð¸Ð¼Ð¸ ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ°Ð¼Ð¸ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ. Ð ÑÐ¾Ð´Ðµ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð±ÑÐ»Ð¸ ÑÐµÑÐµÐ½Ñ ÑÐ»ÐµÐ´ÑÑÑÐ¸Ðµ Ð·Ð°Ð´Ð°ÑÐ¸.
1. ÐÐ·ÑÑÐµÐ½Ð¸Ðµ Ð¾ÑÐ¾Ð±ÐµÐ½Ð½Ð¾ÑÑÐµÐ¹ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ñ Ð¼Ð½Ð¾Ð³Ð¸Ð¼Ð¸ ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ°Ð¼Ð¸, Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ðµ Ð³ÑÐ°ÑÐ° Ð´Ð¸ÑÐºÑÑÑÐ° Ð¸ Ð¾Ð³ÑÐ°Ð½Ð¸ÑÐµÐ½Ð¸Ð¹ Ð½Ð° ÐµÐ³Ð¾ ÑÑÑÑÐºÑÑÑÑ.
2. ÐÑÐ±Ð¾Ñ Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ Ð¿Ð¾Ð´ÑÐ¾Ð´ÑÑÐµÐ³Ð¾ ÑÐ¿Ð¾ÑÐ¾Ð±Ð° Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ ÑÐ»Ð¾Ð² ÐºÐ¾ÑÐ¿ÑÑÐ° Ð² Ð²ÐµÐºÑÐ¾ÑÐ½Ð¾Ð¹ ÑÐ¾ÑÐ¼Ðµ.
3. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ Ð»Ð¾ÐºÐ°Ð»ÑÐ½Ð¾Ð³Ð¾ Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ Ð´Ð»Ñ ÐºÐ°Ð¶Ð´Ð¾Ð¹ ÑÐµÐ¿Ð»Ð¸ÐºÐ¸ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð°, ÐºÐ¾Ð´Ð¸ÑÑÑÑÐ¸Ð¹ Ð¿Ð¾ÑÐ»ÐµÐ´Ð¾Ð²Ð°ÑÐµÐ»ÑÐ½Ð¾ÑÑÑ ÑÐ»Ð¾Ð² Ð² Ð¿ÑÐµÐ´Ð»Ð¾Ð¶ÐµÐ½Ð¸Ð¸ Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ.
4. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÐµÐ»Ð¸, Ð¿Ð¾ÑÐ»ÐµÐ´Ð¾Ð²Ð°ÑÐµÐ»ÑÐ½Ð¾ Ð²ÑÑÐ¸ÑÐ»ÑÑÑÐµÐ¹ Ð½Ð°Ð»Ð¸ÑÐ¸Ðµ ÑÐ²ÑÐ·Ð¸ Ð¼ÐµÐ¶Ð´Ñ ÑÐµÐ¿Ð»Ð¸ÐºÐ°Ð¼Ð¸ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð°.
5. Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÐµÐ»Ð¸, Ð²ÑÑÐ¸ÑÐ»ÑÑÑÐµÐ¹ ÑÐ¸Ð¿ ÑÐ²ÑÐ·Ð¸ Ð¼ÐµÐ¶Ð´Ñ ÑÐµÐ¿Ð»Ð¸ÐºÐ°Ð¼Ð¸.
ÐÑÑÐ¾Ð´Ð½ÑÐ¼Ð¸ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸ Ð´Ð»Ñ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð²ÑÐ±ÑÐ°Ð½ ÐºÐ¾ÑÐ¿ÑÑ STAC - Ð½Ð°Ð±Ð¾Ñ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ð¸Ð· Ð¾Ð½Ð»Ð°Ð¹Ð½ Ð²ÐµÑÑÐ¸Ð¸ Ð¸Ð³ÑÑ The Settlers of Catan. Ð­ÑÐ¾ Ð¸Ð³ÑÐ° ÑÐ¾ Ð¼Ð½Ð¾Ð³Ð¸Ð¼Ð¸ ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ°Ð¼Ð¸, Ð² ÑÐ°ÑÐµ ÐºÐ¾ÑÐ¾ÑÐ¾Ð¹, ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ¸ Ð¼Ð¾Ð³ÑÑ Ð¾Ð±ÑÐ°ÑÑÑÑ Ð½Ð° ÑÐµÐ¼Ñ, Ð½Ðµ ÑÐ²ÑÐ·Ð°Ð½Ð½ÑÐµ Ñ Ð¸Ð³ÑÐ¾Ð²ÑÐ¼ Ð¿ÑÐ¾ÑÐµÑÑÐ¾Ð¼. Ð­ÑÐ¾ Ð´ÐµÐ»Ð°ÐµÑ ÐºÐ¾ÑÐ¿ÑÑ STAC Ð¿Ð¾ÐºÐ°Ð·Ð°ÑÐµÐ»ÑÐ½ÑÐ¼ Ð¿ÑÐ¸Ð¼ÐµÑÐ¾Ð¼ Ð´Ð»Ñ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ñ Ð¼Ð½Ð¾Ð³Ð¸Ð¼Ð¸ ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ°Ð¼Ð¸.
ÐÑÐ»Ð° Ð¸Ð·ÑÑÐµÐ½Ð° ÑÑÑÑÐºÑÑÑÐ° Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ñ Ð¼Ð½Ð¾Ð³Ð¸Ð¼Ð¸ ÑÑÐ°ÑÑÐ½Ð¸ÐºÐ°Ð¼Ð¸ â Ð¾ÑÐ¸ÐµÐ½ÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð½ÑÐ¹ Ð°ÑÐ¸ÐºÐ»Ð¸ÑÐµÑÐºÐ¸Ð¹ Ð³ÑÐ°Ñ. ÐÑÐ½Ð¾Ð²Ð½ÑÐ¼ Ð¾Ð³ÑÐ°Ð½Ð¸ÑÐµÐ½Ð¸ÐµÐ¼, Ð½Ð°ÐºÐ»Ð°Ð´ÑÐ²Ð°ÐµÐ¼ÑÐ¼ Ð½Ð° ÑÑÑÑÐºÑÑÑÑ Ð³ÑÐ°ÑÐ° Ð´Ð¸ÑÐºÑÑÑÐ°, ÑÐ²Ð»ÑÐµÑÑÑ Ð¾ÑÑÑÑÑÑÐ²Ð¸Ðµ Ð½Ð°Ð¿ÑÐ°Ð²Ð»ÐµÐ½Ð½Ð¾Ð¹ ÑÐ²ÑÐ·Ð¸ Ð¾Ñ Ð±Ð¾Ð»ÐµÐµ Ð¿Ð¾Ð·Ð´Ð½Ð¸Ñ ÑÐµÐ¿Ð»Ð¸Ðº Ðº Ð±Ð¾Ð»ÐµÐµ ÑÐ°Ð½Ð½Ð¸Ð¼. Ð ÐµÐ¿Ð»Ð¸ÐºÐ¸ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð² Ð¿Ð¾Ð´ÐµÐ»ÐµÐ½Ñ Ð½Ð° Ð´Ð²Ð° ÑÐ¸Ð¿Ð°: EDU Ð¸ CDU, Ð¿Ð¾ÑÑÐ¾Ð¼Ñ Ð¸ÑÑÐ¾Ð´Ð½ÑÐ¹ Ð³ÑÐ°Ñ Ð´Ð¸ÑÐºÑÑÑÐ° Ð¸Ð¼ÐµÐµÑ Ð´Ð²Ð° ÑÐ¸Ð¿Ð° ÑÐµÐ±ÐµÑ. Ð ÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼, Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÑÑÑÐ¸Ð¹ ÑÐºÐ²Ð¸Ð²Ð°Ð»ÐµÐ½ÑÐ½Ð¾Ðµ Ð¿ÑÐµÐ¾Ð±ÑÐ°Ð·Ð¾Ð²Ð°Ð½Ð¸Ðµ Ðº Ð³ÑÐ°ÑÑ Ñ Ð¾Ð´Ð½Ð¸Ð¼ ÑÐ¸Ð¿Ð¾Ð¼ ÑÐµÐ±ÐµÑ.
ÐÐ±ÑÑÐµÐ½Ð° Ð¼Ð¾Ð´ÐµÐ»Ñ Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ð¹ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½Ð¾Ð¹ ÑÐµÑÐ¸, Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÑÑÑÐ°Ñ Ð´Ð²ÑÐ½Ð°Ð¿ÑÐ°Ð²Ð»ÐµÐ½Ð½ÑÐ¹ Ð°Ð²ÑÐ¾ÑÐ½ÐºÐ¾Ð´ÐµÑ GRU Ð´Ð»Ñ Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ Ð»Ð¾ÐºÐ°Ð»ÑÐ½ÑÑ Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÐµÐ½Ð¸Ð¹ ÑÐµÐ¿Ð»Ð¸Ðº Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð². Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ñ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð´Ð»Ñ Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ Ð³Ð»Ð¾Ð±Ð°Ð»ÑÐ½ÑÑ Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÐµÐ½Ð¸Ð¹ Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð², Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ Ð½Ð°Ð»Ð¸ÑÐ¸Ñ ÑÐ²ÑÐ·Ð¸ Ð¼ÐµÐ¶Ð´Ñ ÑÐµÐ¿Ð»Ð¸ÐºÐ°Ð¼Ð¸ Ð¸ Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ ÑÐ¸Ð¿Ð° ÑÐ²ÑÐ·Ð¸ Ð¼ÐµÐ¶Ð´Ñ ÑÐµÐ¿Ð»Ð¸ÐºÐ°Ð¼Ð¸, Ð¸ÑÐ¿Ð¾Ð»ÑÐ·ÑÑÑÐ¸Ðµ Ð¼ÐµÑÐ¾Ð´Ñ Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ.
Ð ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐµ Ð±ÑÐ»Ð¸ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ñ Ð´Ð»Ñ Ð²Ð¾ÑÑÑÐ°Ð½Ð¾Ð²Ð»ÐµÐ½Ð¸Ñ ÑÑÑÑÐºÑÑÑÑ Ð´Ð¸ÑÐºÑÑÑÐ° ÑÐ¾ Ð·Ð½Ð°ÑÐµÐ½Ð¸ÐµÐ¼ Ð¼ÐµÑÑÐ¸ÐºÐ¸ F_1 0.65 Ð¸ 0.48 Ð´Ð»Ñ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ñ Ð½Ð°Ð»Ð¸ÑÐ¸Ñ ÑÐ²ÑÐ·Ð¸ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ñ ÑÐ¸Ð¿Ð° ÑÐ²ÑÐ·Ð¸ ÑÐ¾Ð¾ÑÐ²ÐµÑÑÑÐ²ÐµÐ½Ð½Ð¾.
ÐÐ¾Ð»ÑÑÐµÐ½Ð½ÑÐµ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÑ Ð¼Ð¾Ð³ÑÑ Ð±ÑÑÑ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ñ Ð´Ð»Ñ Ð±Ð¾Ð»ÐµÐµ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾Ð³Ð¾ ÑÐµÑÐµÐ½Ð¸Ñ Ð´ÑÑÐ³Ð¸Ñ Ð·Ð°Ð´Ð°Ñ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°, ÑÐ°ÐºÐ¸Ñ ÐºÐ°Ðº ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð´Ð¸Ð°Ð»Ð¾Ð³Ð¾Ð²ÑÑ Ð°ÑÑÐ¸ÑÑÐµÐ½ÑÐ¾Ð², ÑÐµÑÐµÐ²ÑÑ Ð¸Ð½ÑÐµÑÑÐµÐ¹ÑÐ¾Ð², Ð° ÑÐ°ÐºÐ¶Ðµ Ð´Ð»Ñ Ð¸Ð½ÑÐ¾ÑÐ¼Ð°ÑÐ¸Ð¾Ð½Ð½Ð¾Ð³Ð¾ Ð¿Ð¾Ð¸ÑÐºÐ° Ð¸ Ð¸Ð·Ð²Ð»ÐµÑÐµÐ½Ð¸Ñ Ð¸Ð½ÑÐ¾ÑÐ¼Ð°ÑÐ¸Ð¸.
description: The theme of the final qualification work: "Development of a system for extracting a description of the structure of discourse from dialogue texts."
This work is devoted to restoring the structure of discourse in dialogs with many participants using machine learning methods. Tasks that were solved during the study:
1. The study of the features of dialogues with many participants, the study of the graph of discourse and the limitations on its structure.
2. The choice of the most suitable way to represent the words of the corpus in vector form.
3. Development of a local representation calculation for each dialogue replica, encoding a sequence of words in a sentence based on machine learning methods.
4. Development of a model that sequentially calculates the presence of communication between dialogue replicas.
5. Development of a model that calculates the type of connection between replicas.
The STAC corpus, a set of dialogs from the online version of the Settlers of Catan game was chosen as the main dataset. Settlers of Catan is a board game with multiple participants that involves trading between players. In the online adaptation of a game, the trading takes place in a chat, which the players also use to discuss topics not related to the gameplay. The diversity of topics and the rich structure of communication in the chat logs make STAC a representative corpus of dialogues with multiple participants.
The discourse structure is usually described as a directed acyclic graph whose vertices correspond to utterances and edges denote discourse relations. The structure of a discourse graph is subject to the chronological ordering constraint: no edge can go from a later utterance to an earlier one. Dialogue utterances are divided into two types: EDU and CDU, so the original discourse graph has two types of edges. We implemented an algorithm that transforms it into a graph that contains EDU vertices only.
A GRU bidirectional auto-encoder has been trained for computing local representations of utterances. Models have been developed for calculating global representations of dialogs, calculating the presence of communication between replicas, and calculating the type of communication between replicas using machine learning methods.
As a result, algorithms were developed for reconstructing the discourse structure with the F_1 metrics of 0.65 and 0.48 to determine the presence of a connection, determining the type of connection, respectively 
The results can be used to more effectively solve other problems of natural language processing, such as the development of interactive assistants, speech interfaces, as well as for information retrieval and information retrieval.