title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð½Ð¾Ð¹ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸ ÑÐ¸ÑÑÐµÐ¼Ñ Ð¿Ð¾Ð½Ð¸Ð¼Ð°Ð½Ð¸Ñ ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ° Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¼Ð°Ð³Ð¸ÑÑÑÐ°
titleType: Subtitle
title: Development of a software implementation of a natural language understanding system based on neural networks
Creators: name: ÐÐ»Ñ ÐÐ»Ð¸, ÐÐ¾Ð°Ð·
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2020
Subjects: Ð¿Ð¾Ð½Ð¸Ð¼Ð°Ð½Ð¸Ñ ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°
Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸
RNNs
CNNs
ÑÐ¾Ð²Ð¼ÐµÑÑÐ½Ð°Ñ Ð¼Ð¾Ð´ÐµÐ»Ñ
Ð·Ð°Ð¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ðµ ÑÐ»Ð¾ÑÐ¾Ð²
Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ðµ ÑÐµÐ»Ð¸
natural language understanding
neural networks
joint model
slot filling
intent detection

description: Ð ÑÑÐ¾Ð¹ ÑÐ°Ð±Ð¾ÑÐµ Ð±ÑÐ»Ð¾ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¾ ÑÐ¾Ð²Ð¼ÐµÑÑÐ½Ð¾Ðµ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°-
Ð½Ð¸Ðµ Ð³Ð¸Ð±ÑÐ¸Ð´Ð½ÑÑ ÑÐ²ÐµÑÑÐ¾ÑÐ½ÑÑ ÑÐµÑÐµÐ¹ Ð¸ ÑÐµÑÐµÐ¹ LSTM (Long short-term
memory) Ð´Ð»Ñ ÑÐµÑÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°Ñ Ð·Ð°Ð¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ ÑÐ»Ð¾ÑÐ¾Ð² Ð¸ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ñ
ÑÐµÐ»Ð¸ Ð¿ÑÐ¸ Ð°Ð½Ð°Ð»Ð¸Ð·Ðµ Ð¿ÑÐµÐ´Ð»Ð¾Ð¶ÐµÐ½Ð¸Ð¹ ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°. ÐÑÐ»Ð° Ð¿ÑÐµÐ´-
Ð»Ð¾Ð¶ÐµÐ½Ð° Ð½Ð¾Ð²Ð°Ñ Ð¼Ð¾Ð´ÐµÐ»Ñ, ÐºÐ¾Ð¼Ð±Ð¸Ð½Ð¸ÑÑÑÑÐ°Ñ ÑÐ²ÐµÑÑÐ¾ÑÐ½ÑÐµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ
ÑÐµÑÐ¸ Ð¸ ÑÐµÐºÑÑÑÐµÐ½ÑÐ½ÑÐµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸, Ð² ÑÐ°Ð¼ÐºÐ°Ñ ÐºÐ¾ÑÐ¾ÑÐ¾Ð¹ ÑÐ²ÐµÑÑÐ¾Ñ-
Ð½ÑÐµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸ Ð¾Ð±Ð½Ð°ÑÑÐ¶Ð¸Ð²Ð°ÑÑ ÑÐ»Ð¾Ð¶Ð½ÑÐµ Ð¿ÑÐ¸Ð·Ð½Ð°ÐºÐ¸ Ð²Ð¾ Ð²ÑÐ¾Ð´-
Ð½ÑÑ Ð¿Ð¾ÑÐ»ÐµÐ´Ð¾Ð²Ð°ÑÐµÐ»ÑÐ½Ð¾ÑÑÑÑ, Ð¼ÐµÑÐ¾Ð´Ð¾Ð¼ Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸Ñ ÑÐ¸Ð»ÑÑÑÐ¾Ð² Ðº ÐºÐ°Ð´-
ÑÐ°Ð¼ ÑÑÐ¸Ñ Ð²ÑÐ¾Ð´Ð¾Ð², Ð° ÑÐµÐºÑÑÑÐµÐ½ÑÐ½ÑÐµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸ Ð¾ÑÑÐ»ÐµÐ¶Ð¸Ð²Ð°ÑÑ
Ð´Ð¾Ð»Ð³Ð¾ÑÑÐ¾ÑÐ½ÑÐµ Ð¸ ÐºÑÐ°ÑÐºÐ¾ÑÑÐ¾ÑÐ½ÑÐµ Ð·Ð°Ð²Ð¸ÑÐ¸Ð¼Ð¾ÑÑÐ¸ Ð²Ð¾ Ð²ÑÐ¾Ð´Ð½ÑÑ Ð¿Ð¾ÑÐ»Ðµ-
Ð´Ð¾Ð²Ð°ÑÐµÐ»ÑÐ½Ð¾ÑÑÑÑ. Ð ÑÐ°Ð¼ÐºÐ°Ñ ÑÐ°Ð±Ð¾ÑÑ ÑÐ°ÐºÐ¶Ðµ Ð±ÑÐ»Ð¾ Ð¿ÑÐ¾Ð°Ð½Ð°Ð»Ð¸Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ð¾
Ð¸ Ð¾Ð±Ð¾ÑÐ½Ð¾Ð²Ð°Ð½Ð¾ Ð¿Ð¾ÑÑÑÐ¾ÐµÐ½Ð¸Ðµ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð´Ð»Ñ ÑÐ¾Ð²Ð¼ÐµÑÑÐ½Ð¾Ð³Ð¾ Ð·Ð°Ð¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ
ÑÐ»Ð¾ÑÐ¾Ð² Ð¸ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ñ ÑÐµÐ»Ð¸, Ð¿Ð¾ÑÐºÐ¾Ð»ÑÐºÑ ÑÑÐ¸ÑÐ°ÐµÑÑÑ, ÑÑÐ¾ Ð¼ÐµÐ¶Ð´Ñ ÑÐµ-
Ð»ÑÑ Ð¸ ÑÐµÐ¼Ð°Ð½ÑÐ¸ÑÐµÑÐºÐ¸Ð¼Ð¸ ÑÐ»Ð¾ÑÐ°Ð¼Ð¸ ÑÑÑÐµÑÑÐ²ÑÐµÑ ÑÐµÑÐ½Ð°Ñ ÑÐ²ÑÐ·Ñ. Ð¡Ð¾Ð²-
Ð¼ÐµÑÑÐ½Ð°Ñ Ð¼Ð¾Ð´ÐµÐ»Ñ Ð¼Ð¾Ð¶ÐµÑ Ð¾ÑÑÐ°Ð¶Ð°ÑÑ ÑÑÐ¾ Ð¾ÑÐ½Ð¾ÑÐµÐ½Ð¸Ðµ, Ð°Ð½Ð°Ð»Ð¸Ð·Ð¸ÑÐ¾Ð²Ð°ÑÑ
ÐµÐ³Ð¾ Ð¸ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°ÑÑ Ð´Ð»Ñ ÑÐ»ÑÑÑÐµÐ½Ð¸Ñ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð² Ð¿ÑÐ¾Ð³Ð½Ð¾Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ.
Ð ÑÐ°Ð±Ð¾ÑÐµ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·ÑÑÑÑÑ Ð½Ð°Ð±Ð¾ÑÑ Ð´Ð°Ð½Ð½ÑÑ ATIS (Airline Travel Infor-
mation System) Ð¸ Snips (Ð³Ð¾Ð»Ð¾ÑÐ¾Ð²Ð°Ñ Ð¿Ð»Ð°ÑÑÐ¾ÑÐ¼Ð° AI Ð´Ð»Ñ Ð¿Ð¾Ð´ÐºÐ»ÑÑÐµÐ½-
Ð½ÑÑ ÑÑÑÑÐ¾Ð¹ÑÑÐ²), Ð´Ð»Ñ Ð¸Ð·Ð¼ÐµÑÐµÐ½Ð¸Ñ Ð¿ÑÐ¾Ð¸Ð·Ð²Ð¾Ð´Ð¸ÑÐµÐ»ÑÐ½Ð¾ÑÑÐ¸ Ð¸ ÑÐ¾ÑÐ½Ð¾ÑÑÐ¸
Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð¸ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ Ð¸Ñ Ñ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ°Ð¼Ð¸ Ð´ÑÑÐ³Ð¸Ñ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹. ÐÐ¾ ÑÐµ-
Ð·ÑÐ»ÑÑÐ°ÑÐ°Ð¼ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ»Ð¸ Ð¿Ð¾Ð»ÑÑÐµÐ½Ñ Ð»ÑÑÑÐ¸Ðµ Ð·Ð½Ð°ÑÐµÐ½Ð¸Ñ
Ð¼ÐµÑÑÐ¸ÐºÐ¸ f1-score Ð² Ð·Ð°Ð´Ð°ÑÐµ Ð·Ð°Ð¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ð¸ ÑÐ»Ð¾ÑÐ¾Ð² Ð´Ð»Ñ Ð½Ð°Ð±Ð¾ÑÐ° Ð´Ð°Ð½Ð½ÑÑ
ATIS (98,21) Ð¸ Ð±Ð»Ð¸Ð·ÐºÐ¸Ðµ Ðº Ð»ÑÑÑÐ¸Ð¼ Ð·Ð½Ð°ÑÐµÐ½Ð¸ÑÐ¼ ÑÐ¾ÑÐ½Ð¾ÑÑÐ¸ Ð² Ð·Ð°Ð´Ð°ÑÐµ
Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ð¸ ÑÐµÐ»Ð¸ Ð´Ð»Ñ Ð½Ð°Ð±Ð¾ÑÐ¾Ð² Ð´Ð°Ð½Ð½ÑÑ ATIS (98,12) Ð¸ SNIPs
(98,42) Ð¿Ð¾ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ Ñ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ°Ð¼Ð¸, Ð½Ð°Ð¹Ð´ÐµÐ½Ð½ÑÐ¼Ð¸ Ð² Ð»Ð¸ÑÐµÑÐ°ÑÑÑÐµ.
description: We investigate the usage of hybrid convolutional and long
short-term memory networks for joint slot filling and intent detection
in natural language understanding. We propose a novel model that
combines between convolutional neural networks, for their ability to
detect complex features in the input sequences by applying filters to
frames of these inputs, and recurrent neural networks taking in account
the fact that they can keep track of the long and short term dependen-
cies in the input sequences. We choose to build a model for joint slot
filling and intent detection, because we believe that there is a strong
relation between the intent and the semantic slots. A joint model can
reflect this relation, figure it out and make use of it to enhance the
prediction results. We use the Airline Travel Information System
(ATIS) and Snips (an AI voice platform for connected devices) da-
tasets to measure the performance of our models and compare them
with the results of other models as these datasets have become the
most popular datasets for natural language understanding problem. By
the results of realization this work, we got the best f1-score values in
slot filling task for ATIS dataset (98.21) and close to the best accuracy
values in intent detection task for ATIS (98.12) and SNIPs (98.42) da-
tasets.