title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¸ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ðµ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ SQL ÐºÐ¾Ð¼Ð°Ð½Ð´ Ð½Ð° Big Data
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Development and research of optimization methods for SQL commands on Big Data
Creators: name: ÐÑÐ»ÑÐµÐ², ÐÐ»ÐµÐºÑÐ°Ð½Ð´Ñ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2021
Subjects: Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ñ
SQL-Ð·Ð°Ð¿ÑÐ¾ÑÑ
Big Data
Hive
Oracle Database
Oracle Big Data Lite
optimization
SQL queries

description: Ð Ð´Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÐµ Ð±ÑÐ»Ð¸ ÑÐ°ÑÑÐ¼Ð¾ÑÑÐµÐ½Ñ ÑÑÐµÑÑ Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸Ñ Big Data Ð² Ð¶Ð¸Ð·Ð½Ð¸, Ð° Ð¸Ð¼ÐµÐ½Ð½Ð¾ ÑÐ¾Ð·Ð´Ð°Ð½Ð¸Ðµ Ð¿ÑÐ¾Ð³Ð½Ð¾Ð·Ð½ÑÑ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹, Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ñ Ð±Ð¸Ð·Ð½ÐµÑ-Ð¿ÑÐ¾ÑÐµÑÑÐ¾Ð², Ð¿ÐµÑÑÐ¾Ð½Ð°Ð»ÑÐ½Ð°Ñ Ð¾ÑÐµÐ½ÐºÐ°, ÑÐ»ÑÑÑÐµÐ½Ð¸Ðµ Ð·Ð´ÑÐ°Ð²Ð¾Ð¾ÑÑÐ°Ð½ÐµÐ½Ð¸Ñ Ð¸ Ð¾Ð±ÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð³Ð¾ Ð·Ð´Ð¾ÑÐ¾Ð²ÑÑ, ÑÐ»ÑÑÑÐµÐ½Ð¸Ðµ Ð½Ð°ÑÐºÐ¸, Ð¿Ð¾Ð²ÑÑÐµÐ½Ð¸Ðµ Ð±ÐµÐ·Ð¾Ð¿Ð°ÑÐ½Ð¾ÑÑÐ¸ Ð¸ Ð¿ÑÐ°Ð²Ð¾Ð¿Ð¾ÑÑÐ´ÐºÐ°, ÑÐ¼Ð½ÑÐµ Ð³Ð¾ÑÐ¾Ð´Ð° Ð¸ ÑÐ¸Ð½Ð°Ð½ÑÐ¾Ð²Ð°Ñ ÑÐ¾ÑÐ³Ð¾Ð²Ð»Ñ. ÐÑÐ»Ð¸ ÑÐ°ÑÑÐ¼Ð¾ÑÑÐµÐ½Ñ Ð¿ÑÐ±Ð»ÐµÐ¼Ñ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Big Data: Ð¾Ð±ÑÑÐ¼, ÑÐ°Ð·ÑÐ¾Ð·Ð½ÐµÐ½Ð½Ð¾ÑÑÑ, Ð½ÐµÑÑÑÑÐºÑÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð½Ð¾ÑÑÑ Ð´Ð°Ð½Ð½ÑÑ, Ð¸Ñ ÑÐºÐ¾ÑÐ¾ÑÑÑ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸, Ð´Ð¾ÑÑÐ¾Ð²ÐµÑÐ½Ð¾ÑÑÑ, Ð¸Ð·Ð¼ÐµÐ½ÑÐ¸Ð²Ð¾ÑÑÑ, Ð²Ð¸Ð·ÑÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð¸ ÑÐµÐ½Ð½Ð¾ÑÑÑ. Ð Ð°ÑÑÐ¼Ð¾ÑÑÐµÐ½Ñ Ð¿Ð¾Ð´ÑÐ¾Ð´Ñ Ðº ÑÐ°Ð±Ð¾ÑÐµ Ñ Big Data: hadoop-ÑÐ¸ÑÑÐµÐ¼Ñ, NoSQL Ð±Ð°Ð·Ñ Ð´Ð°Ð½Ð½ÑÑ, ÑÐµÐ»ÑÑÐ¸Ð¾Ð½Ð½ÑÐµ Ð¡Ð£ÐÐ, Oracle Big Data Lite. ÐÑÐ»Ð¸ Ð¸Ð·ÑÑÐµÐ½Ñ Ð¿Ð¾Ð´ÑÐ¾Ð´Ñ Ðº Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð·Ð°Ð¿ÑÐ¾ÑÐ¾Ð² Ð² Ð¾Ð´Ð½Ð¾ÑÐ¾Ð´Ð½Ð¾Ð¹ (ÑÐ°ÐºÐ¸Ñ ÐºÐ°Ðº Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ðµ Ð¸Ð½Ð´ÐµÐºÑÐ¾Ð² Ð¸ ÑÐ¸Ð½ÑÐ°ÐºÑÐ¸ÑÐ° ANSI) Ð¸ Ð³ÐµÑÐµÑÐ¾Ð³ÐµÐ½Ð½Ð¾Ð¹ ÑÑÐµÐ´Ð°Ñ (ÑÐ¾Ð·Ð´Ð°Ð½Ð¸Ðµ Ð°Ð±ÑÑÑÐ°ÐºÑÐ¸Ð¸, Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ðµ Ð¸Ð½Ð´ÐµÐºÑÐ¾Ð² ÑÑÐ°Ð½ÐµÐ½Ð¸Ñ). ÐÐ¾Ð´ÑÐ¾Ð±Ð½Ð¾ Ð¸Ð·ÑÑÐµÐ½Ð° ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð° ÐºÐ¾Ð¼Ð¿Ð°Ð½Ð¸Ð¸ Oracle. ÐÑÐ»Ð° ÑÐ¾ÑÑÐ°Ð²Ð»ÐµÐ½Ð° ÑÑÐµÐ¼Ð° Ð±Ð°Ð·Ñ Ð´Ð°Ð½Ð½ÑÑ, Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ ÐºÐ¾ÑÐ¾ÑÐ¾Ð¹ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ ÑÐµÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð°. ÐÑÑÐ²Ð»ÐµÐ½Ñ ÑÑÑÐµÐºÑÐ¸Ð²Ð½ÑÐµ Ð¼ÐµÑÐ¾Ð´Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ SQL-Ð·Ð°Ð¿ÑÐ¾ÑÐ¾Ð², Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ ÐºÐ¾ÑÐ¾ÑÑÑ ÑÐ¾ÑÑÐ°Ð²Ð»ÐµÐ½Ð° Ð¼ÐµÑÐ¾Ð´Ð¸ÐºÐ° Ð¿Ð¾ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾Ð¼Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ñ ÑÐ¿Ð¾ÑÐ¾Ð±Ð° Ð²Ð·Ð°Ð¸Ð¼Ð¾Ð´ÐµÐ¹ÑÑÐ²Ð¸Ñ Ð¼ÐµÐ¶Ð´Ñ NoSQL Ð¸ SQL Ð±Ð°Ð·Ð°Ð¼Ð¸ Ð´Ð°Ð½Ð½ÑÑ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð²Ð½ÐµÑÐ½Ð¸Ñ ÑÐ°Ð±Ð»Ð¸Ñ. ÐÑÐ» ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½ ÑÐ¾Ð±ÑÑÐ²ÐµÐ½Ð½ÑÐ¹ Ð¿Ð¾Ð´ÑÐ¾Ð´ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ SQL-Ð·Ð°Ð¿ÑÐ¾ÑÐ¾Ð², Ð±ÑÐ»Ð° Ð´Ð°Ð½Ð° Ð¾ÑÐµÐ½ÐºÐ° ÐµÐ³Ð¾ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸. ÐÑÐµÐ´Ð»Ð¾Ð¶ÐµÐ½Ñ ÑÐ°Ð³Ð¸ ÑÐ°Ð·Ð²Ð¸ÑÐ¸Ñ Ð´Ð°Ð½Ð½Ð¾Ð³Ð¾ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð°.
description: In this paper, the areas of application of Big Data in life were considered, namely, the creation of predictive models, optimization of business processes, personal assessment, improvement of healthcare and public health, improvement of science, improvement of safety and law and order, smart cities and financial trade. The problems of Big Data processing were considered: volume, fragmentation, unstructured data, their processing speed, reliability, variability, visualization and value. The approaches to working with Big Data are considered: hadoop-systems, NoSQL databases, relational DBMS, Oracle Big Data Lite. Approaches to optimizing queries in homogeneous (such as using indexes and ANSI syntax) and heterogeneous environments (creating abstractions, using storage indices) were studied. The implementation of the approach of the Oracle company has been studied in detail. A database schema was drawn up and used to test the approach. The effective methods of optimizing SQL queries are revealed, on the basis of which a methodology for the effective use of the method of interaction between NoSQL and SQL databases using external tables has been compiled. A proprietary approach to optimizing SQL queries was developed, and its effectiveness was assessed. Steps for the development of this approach are proposed.