title: ÐÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ðµ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸Ñ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹ Ð² Ð·Ð°Ð´Ð°ÑÐµ Ð¿ÐµÑÐµÐ²Ð¾Ð´Ð° ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ° Ð² SQL
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Research of the effectiveness of neural networks in the task of translating natural language into SQL
Creators: name: ÐÐ°ÑÐµÐ½Ð¾Ð², ÐÐ²Ð³ÐµÐ½Ð¸Ð¹
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2021
Subjects: NL2SQL
Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸
GPT-3
BERT
T5
ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð°Ñ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ° ÑÐ·ÑÐºÐ°
ÑÐµÐºÑÑÑÐµÐ½ÑÐ½ÑÐµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸
ÑÑÐ°Ð½ÑÑÐ¾ÑÐ¼ÐµÑ
machine learning
neural networks
natural language processing
recurrent neural networks
transformer

description: Ð Ð´Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÐµ ÑÐ°ÑÑÐ¼Ð°ÑÑÐ¸Ð²Ð°ÐµÑÑÑ ÑÐµÑÐµÐ½Ð¸Ðµ Ð·Ð°Ð´Ð°ÑÐ¸ Ð¿ÐµÑÐµÐ²Ð¾Ð´Ð° ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ° Ð² SQL Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ ÑÐ¾Ð²ÑÐµÐ¼ÐµÐ½Ð½ÑÑ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑ GPT-3, BERT, T5. ÐÑÐ¾Ð²Ð¾Ð´Ð¸ÑÑÑ ÑÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· ÑÑÐ¸Ñ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑ Ð¿ÑÐ¸ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ð¸ Ð´Ð²ÑÑ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð¾Ð²: Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ sketch-based Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÑ, Ð¿ÑÐ¸ ÐºÐ¾ÑÐ¾ÑÐ¾Ð¹ ÐºÐ¾Ð½ÐµÑÐ½ÑÐ¹ Ð·Ð°Ð¿ÑÐ¾Ñ Ð½Ðµ Ð³ÐµÐ½ÐµÑÐ¸ÑÑÐµÑÑÑ Ð¿Ð¾Ð»Ð½Ð¾ÑÑÑÑ, Ð° Ð·Ð°Ð¿Ð¾Ð»Ð½ÑÑÑÑÑ Â«ÑÐ»Ð¾ÑÑÂ» Ð¸ Ñ Ð³ÐµÐ½ÐµÑÐ°ÑÐ¸ÐµÐ¹ ÐºÐ¾Ð½ÐµÑÐ½Ð¾Ð³Ð¾ Ð·Ð°Ð¿ÑÐ¾ÑÐ° Ð½Ð° SQL text-to-text. ÐÐ»Ñ ÑÐµÑÐµÐ½Ð¸Ñ Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½Ð¾Ð¹ Ð·Ð°Ð´Ð°ÑÐ¸ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ ÑÐ°Ð·Ð»Ð¸ÑÐ½ÑÑ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹ Ð´Ð»Ñ Ð½Ð°ÑÐ°Ð»Ð° Ð±ÑÐ»Ð¸ ÑÐ°ÑÑÐ¼Ð¾ÑÑÐµÐ½Ñ ÑÑÑÐµÑÑÐ²ÑÑÑÐ¸Ðµ ÑÐµÑÐµÐ½Ð¸Ñ. ÐÑÐ»Ð¸ Ð¿Ð¾Ð´ÑÐ¾Ð±Ð½Ð¾ Ð¸Ð·ÑÑÐµÐ½Ñ ÑÑÑÑÐ¾Ð¹ÑÑÐ²Ð° Ð°ÑÑÐ¸ÑÐµÐºÑÑÑ GPT-3, T5 Ð¸ BERT Ð´Ð»Ñ Ð¿Ð¾ÑÐ»ÐµÐ´ÑÑÑÐµÐ¹ Ð¸Ð½ÑÐµÐ³ÑÐ°ÑÐ¸Ð¸ Ð² Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð´Ð»Ñ ÑÐµÑÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°ÑÐ¸ NL2SQL. ÐÐ»Ñ sketch-based Ð¿Ð¾Ð´ÑÐ¾Ð´Ð° Ð·Ð° Ð±ÐµÐ¹Ð·Ð»Ð°Ð¹Ð½ Ð±ÑÐ»Ð° Ð²Ð·ÑÑÐ° Ð¼Ð¾Ð´ÐµÐ»Ñ SQLova, ÐºÐ¾ÑÐ¾ÑÐ°Ñ Ð±ÑÐ»Ð° Ð·Ð°ÑÐµÐ¼ Ð¿ÐµÑÐµÐ¿Ð¸ÑÐ°Ð½Ð° Ð´Ð»Ñ Ð²Ð½ÐµÑÐµÐ½Ð¸Ñ Ð´Ð¾Ð¿Ð¾Ð»Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÑ Ð¸Ð·Ð¼ÐµÐ½ÐµÐ½Ð¸Ð¹. ÐÑÐ»Ð¸ Ð¾Ð±ÑÑÐµÐ½Ñ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ GPT-3, T5 Ð¸ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½ ÑÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð². Ð text-to-text Ð¿Ð¾Ð´ÑÐ¾Ð´Ðµ Ð·Ð° Ð¾ÑÐ½Ð¾Ð²Ð½ÑÑ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð»Ð°ÑÑ Ð¼Ð¾Ð´ÐµÐ»Ñ T5, ÐºÐ¾ÑÐ¾ÑÐ°Ñ Ð±ÐµÐ· Ð´Ð¾Ð¿Ð¾Ð»Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÑ ÑÐµÑÐµÐ½Ð¸Ð¹ Ð¿Ð¾ÐºÐ°Ð·Ð°Ð»Ð° ÑÑÐ°Ð·Ñ ÑÐ¾ÑÐ¾ÑÐ¸Ð¹ ÑÐµÐ·ÑÐ»ÑÑÐ°Ñ. ÐÐ°Ð»ÐµÐµ Ð¾Ð½Ð° Ð±ÑÐ»Ð° ÑÐ»ÑÑÑÐµÐ½Ð° Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ gated extraction network, Ð³ÐµÐ½ÐµÑÐ°ÑÐ¸ÐµÐ¹ Ð²ÑÐ¾ÑÐ¸ÑÐ½ÑÑ Ð´Ð°Ð½Ð½ÑÑ. Ð¢Ð°ÐºÐ°Ñ Ð¶Ðµ Ð¼Ð¾Ð´ÐµÐ»Ñ Ð±ÑÐ»Ð° Ð¾Ð±ÑÑÐµÐ½Ð° Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÑ GPT-3, Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð².
description: This paper considers the solution of the problem of natural language translation in SQL using modern architectures GPT-3, BERT, T5.A comparative analysis of these architectures is carried out using two approaches: using a sketch-based architecture, in which the final query is not generated completely, but the "slots" are filled, and with the generation of the final query in SQL text-to-text. To solve the obtained problem of studying the effectiveness of various neural networks, we first considered the existing solutions. The devices of the GPT-3, T5, and BERT architectures were studied in detail for subsequent integration into models for solving the NL2SQL problem. For the sketch-based approach, the SQLova model was taken as the baseline, which was then rewritten to make additional changes. Models were trained using GPT-3, T5 and a comparative analysis of the results was carried out. In the text-to-text approach, the T5 model was used for the main one, which immediately showed a good result without additional solutions. It was further improved with the gated extraction network, the generation of secondary data. The same model was trained using the GPT-3 architecture, and the results were compared.