title: ÐÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ðµ Ð¸ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½ÑÐ°Ð»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· ÑÑÐµÐ´ÑÑÐ² ÑÐ°Ð±Ð¾ÑÑ Ñ Ð±Ð¾Ð»ÑÑÐ¸Ð¼Ð¸ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸ Ð² Ð²ÐµÐ±-Ð¿ÑÐ¸Ð»Ð¾Ð¶ÐµÐ½Ð¸ÑÑ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Research and experimental analysis of big data tools in web applications
Creators: name: Ð¤ÑÐ¾Ð»Ð¾Ð², ÐÐµÐ¾ÑÐ³Ð¸Ð¹
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2024
Subjects: Ð±Ð¾Ð»ÑÑÐ¸Ðµ Ð´Ð°Ð½Ð½ÑÐµ
apache spark
hadoop
Ð¿Ð°ÐºÐµÑÐ½ÑÐµ Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ
Ð¿Ð¾ÑÐ¾ÐºÐ¾Ð²ÑÐµ Ð²ÑÑÐ¸ÑÐ»ÐµÐ½Ð¸Ñ
big data
batch computing
stream computing

description: Ð¦ÐµÐ»Ñ ÑÐ°Ð±Ð¾ÑÑ Ð·Ð°ÐºÐ»ÑÑÐ°ÐµÑÑÑ Ð² Ð²ÑÐ±Ð¾ÑÐµ, Ð¸ Ð¿Ð¾ÑÐ»ÐµÐ´ÑÑÑÐµÐ¼ Ð°Ð½Ð°Ð»Ð¸Ð·Ðµ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÐ½ÑÑ Ð¾ÑÐ¾Ð±ÐµÐ½Ð½Ð¾ÑÑÐµÐ¹ ÑÑÐµÐ´ÑÑÐ² ÑÐ°Ð±Ð¾ÑÑ Ñ Ð±Ð¾Ð»ÑÑÐ¸Ð¼Ð¸ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸, Ð° ÑÐ°ÐºÐ¶Ðµ Ð¾ÑÐµÐ½ÐºÐ° Ð²ÑÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ ÑÑÐµÐ´ÑÑÐ² ÑÐ°Ð±Ð¾ÑÑ Ñ Ð±Ð¾Ð»ÑÑÐ¸Ð¼Ð¸ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸. Ð ÑÐ°Ð¼ÐºÐ°Ñ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ»Ð¸ Ð²ÑÐ±ÑÐ°Ð½Ñ Ð½Ð°Ð¸Ð±Ð¾Ð»ÐµÐµ Ð¿Ð¾Ð¿ÑÐ»ÑÑÐ½ÑÐµ ÑÑÐµÐ´ÑÑÐ²Ð° Ð¿Ð°ÐºÐµÑÐ½Ð¾Ð¹ Ð¸ Ð¿Ð¾ÑÐ¾ÐºÐ¾Ð²Ð¾Ð¹ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ. ÐÑÐ»Ð¸ ÑÑÐ°Ð²Ð½ÐµÐ½Ñ Ð¸Ñ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÐ½ÑÐµ Ð¾ÑÐ¾Ð±ÐµÐ½Ð½Ð¾ÑÑÐ¸ Ð¸ Ð¿Ð¾Ð´ÑÐ¾Ð´ Ðº ÑÐ¾ÑÐ¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Ð¸ Ð¸ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°Ñ. ÐÐ¾ÑÐ»Ðµ ÑÐµÐ³Ð¾ Ð±ÑÐ»Ð¾ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð¾ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½ÑÐ°Ð»ÑÐ½Ð¾Ðµ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ Ð²ÑÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ð¸Ð¼Ð¸ Ð¿Ð¾ÑÐ¾ÐºÐ¾Ð²Ð¾Ð¹ Ð¸ Ð¿Ð°ÐºÐµÑÐ½Ð¾Ð¹ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ. Ð ÑÐ°Ð¼ÐºÐ°Ñ Ð¿Ð¾ÑÐ¾ÐºÐ¾Ð²Ð¾Ð¹ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ ÑÑÐ°Ð²Ð½Ð¸Ð²Ð°Ð»Ð¸ÑÑ Apache Flink Ð¸ Spark Streaming. ÐÐ±Ð° Ð¿ÑÐ¾Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ¸ÑÐ¾Ð²Ð°Ð»Ð¸ Ð´Ð¾ÑÑÐ¾Ð¹Ð½ÑÐµ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÑ, Ð¸ Ð¾Ð±Ð° Ð¼Ð¾Ð³ÑÑ ÑÑÐ¸ÑÐ°ÑÑÑÑ Ð°ÐºÑÑÐ°Ð»ÑÐ½ÑÐ¼Ð¸ Ð¸Ð½ÑÑÑÑÐ¼ÐµÐ½ÑÐ°Ð¼Ð¸ Ð´Ð»Ñ Ð¿Ð¾ÑÐ¾ÐºÐ¾Ð²Ð¾Ð¹ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ. Ð¢ÐµÐ¼ Ð½Ðµ Ð¼ÐµÐ½ÐµÐµ Apache Flink Ð¿ÑÐ¾Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ¸ÑÐ¾Ð²Ð°Ð» Ð±Ð¾Ð»ÐµÐµ ÑÑÐ°Ð±Ð¸Ð»ÑÐ½ÑÑ Ð¸ Ð·Ð½Ð°ÑÐ¸ÑÐµÐ»ÑÐ½Ð¾ Ð¼ÐµÐ½ÑÑÑÑ Ð°Ð±ÑÐ¾Ð»ÑÑÐ½ÑÑ Ð·Ð°Ð´ÐµÑÐ¶ÐºÑ, Ð½ÐµÐ¶ÐµÐ»Ð¸ Spark Streaming. Ð ÑÐ°Ð¼ÐºÐ°Ñ Ð¿Ð°ÐºÐµÑÐ½Ð¾Ð¹ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ ÑÑÐ°Ð²Ð½Ð¸Ð²Ð°Ð»Ð¸ÑÑ ÑÑÐµÐ¹Ð¼Ð²Ð¾ÑÐºÐ¸ Spark, Hive on Tez Ð¸ Hadoop Map Reduce. ÐÑÐ» ÑÐ´ÐµÐ»Ð°Ð½ Ð²ÑÐ²Ð¾Ð´, ÑÑÐ¾ Spark Ð½Ð° ÑÐµÐ³Ð¾Ð´Ð½ÑÑÐ½Ð¸Ð¹ Ð´ÐµÐ½Ñ Ð¿ÑÐµÐ²Ð¾ÑÑÐ¾Ð´Ð¸Ñ Ð´ÑÑÐ³Ð¸Ðµ ÑÑÐ°Ð²Ð½Ð¸Ð²Ð°ÐµÐ¼ÑÐµ Ð¸Ð½ÑÑÑÑÐ¼ÐµÐ½ÑÑ Ð² Ð¾ÑÐ½Ð¾ÑÐµÐ½Ð¸Ð¸ Ð²ÑÐµÐ¼ÐµÐ½Ð½Ð¾Ð¹ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸. Hive on Tez Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ¸ÑÑÐµÑ Ð½ÐµÐ±Ð¾Ð»ÑÑÐ¾Ðµ Ð¾ÑÑÑÐ°Ð²Ð°Ð½Ð¸Ðµ Ð¿Ð¾ ÑÑÑÐµÐºÑÐ¸Ð²Ð½Ð¾ÑÑÐ¸ Ð²ÑÐ¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ñ Ð·Ð°Ð´Ð°Ñ Ð¿Ð°ÐºÐµÑÐ½Ð¾Ð¹ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ Ð¿Ð¾ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ñ ÑÐ¾ Spark, Ð¸ Ð²Ð²Ð¸Ð´Ñ ÑÐ´Ð¾Ð±ÑÑÐ²Ð° Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ñ HiveQL ÑÐ¸Ð½ÑÐ°ÐºÑÐ¸ÑÐ° ÑÐ°Ðº Ð¿ÑÐ¸Ð·Ð½Ð°ÐµÑÑÑ ÑÑÑÐµÐºÑÐ¸Ð²Ð½ÑÐ¼ Ð¸ Ð°ÐºÑÑÐ°Ð»ÑÐ½ÑÐ¼ Ð¸Ð½ÑÑÑÑÐ¼ÐµÐ½ÑÐ¾Ð¼. Hadoop Map Reduce, Ð½Ð°Ð¿ÑÐ¾ÑÐ¸Ð², Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ¸ÑÑÐµÑ Ð´Ð¾ÑÑÐ°ÑÐ¾ÑÐ½Ð¾ ÑÐ»Ð°Ð±ÑÐµ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÑ Ð¸ Ð½Ðµ ÑÐµÐºÐ¾Ð¼ÐµÐ½Ð´ÑÐµÑÑÑ Ðº Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ñ Ð²Ð²Ð¸Ð´Ñ ÑÑÑÐµÑÑÐ²Ð¾Ð²Ð°Ð½Ð¸Ñ Ð±Ð¾Ð»ÐµÐµ Ð±ÑÑÑÑÑÑ Ð¸ ÑÐ´Ð¾Ð±Ð½ÑÑ Ð°Ð»ÑÑÐµÑÐ½Ð°ÑÐ¸Ð², Ð¿ÐµÑÐµÑÐ¸ÑÐ»ÐµÐ½Ð½ÑÑ Ð²ÑÑÐµ.
description: The purpose of the work is to select and then analyze the architectural features of big data tools, as well as to evaluate the temporal efficiency of big data tools. Within the framework of the work the most popular means of batch and stream data processing were selected. Their architectural features and approach to task generation and execution were compared. After that, an experimental comparison of the temporal efficiency of their execution of streaming and batch data processing was conducted. Apache Flink and Spark Streaming were compared in terms of streaming data processing. Both demonstrated decent results, and both can be considered relevant tools for stream processing. However, Apache Flink demonstrated a more stable and significantly lower absolute latency than Spark Streaming. The Spark, Hive on Tez, and Hadoop Map Reduce frameworks were compared within the batch processing framework. It was concluded that Spark by far outperforms the other compared tools in terms of temporal efficiency. Hive on Tez shows a slight lag in the efficiency of batch processing tasks compared to Spark, and due to the ease of use of the HiveQL syntax is so recognized as an effective and relevant tool. Hadoop Map Reduce, on the contrary, demonstrates rather poor results and is not recommended for use due to the existence of faster and more convenient alternatives listed above.