title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÑÐ»Ñ ÑÐ¸ÑÑÐµÐ¼ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð¸ Ð¿Ð¾Ð´Ð³Ð¾ÑÐ¾Ð²ÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ Ð´Ð»Ñ Ð±Ð¸Ð·Ð½ÐµÑ-Ð°Ð½Ð°Ð»Ð¸ÑÐ¸ÐºÐ¸ Ð² ÑÐ¸Ð½Ð°Ð½ÑÐ¾Ð²Ð¾-ÑÐµÑ Ð½Ð¸ÑÐµÑÐºÐ¾Ð¹ Ð¾ÑÐ³Ð°Ð½Ð¸Ð·Ð°ÑÐ¸Ð¸
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: The data preprocessing system module development for business analytics in the fintech company
Creators: name: Ð¢Ð°Ð¶Ð´Ð°Ð²Ð»ÐµÑÐ¾Ð²Ð°, ÐÐ°Ð»ÐµÑÐ¸Ñ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2020
Subjects: ÑÐ¿ÑÐ°Ð²Ð»ÐµÐ½Ð¸Ðµ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸
Ð±Ð¾Ð»ÑÑÐ¸Ðµ Ð´Ð°Ð½Ð½ÑÐµ
Ð¿Ð¾ÑÐ¾ÐºÐ¾Ð²Ð°Ñ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ°
Ð»ÑÐ¼Ð±Ð´Ð°- Ð°ÑÑ Ð¸ÑÐµÐºÑÑÑÐ°
ÑÐ¸Ð»ÑÑÑÐ°ÑÐ¸Ñ Ð´Ð°Ð½Ð½ÑÑ
flink
scala
data government
big data
stream processing
lambda-architecture
data filtering

description: Ð¢ÐµÐ¼Ð° Ð²ÑÐ¿ÑÑÐºÐ½Ð¾Ð¹ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ: âÐ Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÑÐ»Ñ ÑÐ¸ÑÑÐµÐ¼ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ¸ Ð¸ Ð¿Ð¾Ð´Ð³Ð¾ÑÐ¾Ð²ÐºÐ¸ Ð´Ð°Ð½Ð½ÑÑ Ð´Ð»Ñ Ð±Ð¸Ð·Ð½ÐµÑ- Ð°Ð½Ð°Ð»Ð¸ÑÐ¸ÐºÐ¸ Ð² ÑÐ¸Ð½Ð°Ð½ÑÐ¾Ð²Ð¾-ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¾Ð¹ Ð¾ÑÐ³Ð°Ð½Ð¸Ð·Ð°ÑÐ¸Ð¸â
Ð¦ÐµÐ»ÑÑ ÑÐ°Ð±Ð¾ÑÑ ÑÐ²Ð»ÑÐµÑÑÑ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÑÐ»Ñ ÑÐ¸Ð»ÑÑÑÐ°ÑÐ¸Ð¸ Ð´Ð°Ð½Ð½ÑÑ Ð´Ð»Ñ ÑÐ¸ÑÑÐµÐ¼Ñ ÑÐ¿ÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸ Ð² ÑÐ¸Ð½Ð°Ð½ÑÐ¾Ð²Ð¾- ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¾Ð¹ Ð¾ÑÐ³Ð°Ð½Ð¸Ð·Ð°ÑÐ¸Ð¸. ÐÐ°Ð´Ð°ÑÐ¸, ÐºÐ¾ÑÐ¾ÑÑÐµ ÑÐµÑÐ°Ð»Ð¸ÑÑ Ð² ÑÐ¾Ð´Ðµ Ð½Ð°Ð¿Ð¸ÑÐ°Ð½Ð¸Ñ ÑÐ°Ð±Ð¾ÑÑ: Ð¾Ð±Ð·Ð¾Ñ ÑÑÑÐµÑÑÐ²ÑÑÑÐ¸Ñ Ð¼ÐµÑÐ¾Ð´Ð¸Ðº ÑÐ¿ÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸, Ð°Ð½Ð°Ð»Ð¸Ð· ÑÑÑÐµÑÑÐ²ÑÑÑÐµÐ¹ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÑ ÑÐ¸ÑÑÐµÐ¼Ñ ÑÐ¿ÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸ Ð² ÑÐ¸Ð½Ð°Ð½ÑÐ¾Ð²Ð¾-ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¾Ð¹
Ð¾ÑÐ³Ð°Ð½Ð¸Ð·Ð°ÑÐ¸Ð¸ Ñ ÑÐµÐ»ÑÑ ÐµÐµ ÑÑÐ¾Ð²ÐµÑÑÐµÐ½ÑÑÐ²Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¸ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼Ð¾Ð´ÑÐ»Ñ ÑÐ¸Ð»ÑÑÑÐ°ÑÐ¸Ð¸ Ð´Ð°Ð½Ð½ÑÑ.
Ð Ð³Ð»Ð°Ð²Ðµ 1 Ð¿ÑÐµÐ´ÑÑÐ°Ð²Ð»ÐµÐ½Ð° ÑÐµÐ¾ÑÐµÑÐ¸ÑÐµÑÐºÐ°Ñ Ð±Ð°Ð·Ð°, Ð½ÐµÐ¾Ð±ÑÐ¾Ð´Ð¸Ð¼Ð°Ñ Ð´Ð»Ñ Ð°Ð½Ð°Ð»Ð¸Ð·Ð° Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¿Ð¾ÑÑÑÐ¾ÐµÐ½Ð¸Ñ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑ ÑÐ¸ÑÑÐµÐ¼ ÑÐ¿ÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸, Ð¸ Ð¾Ð´Ð¸Ð½ Ð¸Ð· Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¿Ð¾ÑÑÑÐ¾ÐµÐ½Ð¸Ñ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÑ ÑÐ°ÐºÐ¾Ð¹ ÑÐ¸ÑÑÐµÐ¼Ñ. Ð Ð³Ð»Ð°Ð²Ðµ 2 Ð¾Ð¿Ð¸ÑÐ°Ð½Ð° Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÐ° ÑÐ¸ÑÑÐµÐ¼Ñ ÑÐ¿ÑÐ°Ð²Ð»ÐµÐ½Ð¸Ñ Ð´Ð°Ð½Ð½ÑÐ¼Ð¸ Ð² Ð¸Ð·ÑÑÐ°ÐµÐ¼Ð¾Ð¹ Ð¾ÑÐ³Ð°Ð½Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð¸ ÐµÐµ Ð°Ð½Ð°Ð»Ð¸Ð·. Ð Ð³Ð»Ð°Ð²Ð°Ñ 3, 4 Ð¸ 5 Ð¾Ð¿Ð¸ÑÑÐ²Ð°ÑÑÑÑ Ð¿ÑÐ¾ÐµÐºÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ Ð¼Ð¾Ð´ÑÐ»Ñ ÑÐ¸Ð»ÑÑÑÐ°ÑÐ¸Ð¸ Ð´Ð°Ð½Ð½ÑÑ, ÐµÐ³Ð¾ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¸ ÑÐµÑÑÐ¸ÑÐ¾Ð²Ð°Ð½Ð¸Ðµ ÑÐ¾Ð¾ÑÐ²ÐµÑÑÑÐ²ÐµÐ½Ð½Ð¾.
Ð ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐµ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ» ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½ Ð¸ Ð²Ð½ÐµÐ´ÑÐµÐ½ Ð² ÑÐ¸ÑÑÐµÐ¼Ñ ÑÐµÑÐ²Ð¸Ñ, Ð¿Ð¾Ð·Ð²Ð¾Ð»ÑÑÑÐ¸Ð¹ Ð¾Ð±ÑÐ°Ð±Ð°ÑÑÐ²Ð°ÑÑ Ð¿Ð¾ÑÐ¾Ðº Ð´Ð°Ð½Ð½ÑÑ Ð¸ Ð²ÑÑÐ²Ð»ÑÑÑ Ð´ÑÐ±Ð»Ð¸ÑÑÑÑÐ¸ÐµÑÑ Ð¾Ð±ÑÐµÐºÑÑ Ð´Ð°Ð½Ð½ÑÑ.
description: The subject of the graduate work is âThe module development of the data preprocessing and preparation systems for business analytics in a financial and technical organizationâ
The aim of the work is to develop a data filtering module for the data management system in a financial and technical organization. Tasks that were solved during this work are as follows: overview of existing data management methods, analysis of the existing architecture of the data management system in the financial and technical organization with the aim of its improving and development of the data filtering module.
The theoretical basis necessary for the analysis of methods for constructing architectures of data management systems, and one of the methods for constructing the architecture of such a system are presented in Chapter 1. Chapter 2 describes the architecture of the data management system in the organization under study and its analysis. Chapters 3, 4, and 5 describe the design of the data filtering
module, its development and testing, respectively.
As a result of the work the module for data filtering was
implemented and integrated it the system, that allows to dispose duplicate data objects from a data stream.