title: ÐÐ¾ÑÑÑÐ¾ÐµÐ½Ð¸Ðµ Ð¿Ð¾Ð»Ð½Ð¾ÑÐµÐºÑÑÐ¾Ð²Ð¾Ð³Ð¾ Ð¿Ð¾Ð¸ÑÐºÐ° Ð¿Ð¾ ÑÐµÐµÑÑÑÐ°Ð¼ Ð¾Ð³ÑÐ°Ð½Ð¸ÑÐµÐ½Ð½Ð¾Ð¹ Ð¾Ð±Ð»Ð°ÑÑÐ¸ Ð¾Ð¿ÑÐµÐ´ÐµÐ»ÐµÐ½Ð¸Ñ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¼Ð°Ð³Ð¸ÑÑÑÐ°
titleType: Subtitle
title: Building a full-text search on data sources of a limited scope
Creators: name: Ð¡ÑÑÐ¸ÐºÐ¾Ð², ÐÐ»ÑÑ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2023
Subjects: ÐÐ°Ð·Ñ Ð´Ð°Ð½Ð½ÑÑ
ÐÐ°ÑÐµÐ¼Ð°ÑÐ¸ÑÐµÑÐºÐ¸Ð¹ Ð°Ð½Ð°Ð»Ð¸Ð·
Ð¿Ð¾Ð»Ð½Ð¾ÑÐµÐºÑÑÐ¾Ð²ÑÐ¹ Ð¿Ð¾Ð¸ÑÐº
ÑÐ±Ð¾Ñ Ð¸ Ð¾Ð±ÑÐ°Ð±Ð¾ÑÐºÐ° Ð´Ð°Ð½Ð½ÑÑ
full text search
data collection and processing

description: ÐÐ°Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð¾Ð² Ðº Ð¿Ð¾ÑÑÑÐ¾ÐµÐ½Ð¸Ñ Ð¿Ð¾Ð»Ð½Ð¾ÑÐµÐºÑÑÐ¾Ð²Ð¾Ð³Ð¾ Ð¿Ð¾Ð¸ÑÐºÐ° Ð¿Ð¾ Ð¾Ð³ÑÐ°Ð½Ð¸ÑÐµÐ½Ð½Ð¾Ð¼Ñ Ð½Ð°Ð±Ð¾ÑÑ Ð´Ð°Ð½Ð½ÑÑ. Ð¦ÐµÐ»ÑÑ Ð´Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ ÑÐ²Ð»ÑÐµÑÑÑ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° ÑÐ¸ÑÑÐµÐ¼Ñ Ð¿Ð¾Ð»Ð½Ð¾ÑÐµÐºÑÑÐ¾Ð²Ð¾Ð³Ð¾ Ð¿Ð¾Ð¸ÑÐºÐ° Ð¿Ð¾ ÑÐµÐµÑÑÑÐ°Ð¼ Ð´Ð°Ð½Ð½ÑÑ ÑÐ¾ÑÑÐ¸Ð¹ÑÐºÐ¸Ñ ÐºÐ¾Ð¼Ð¿Ð°Ð½Ð¸Ð¹, Ð°Ð³ÑÐµÐ³Ð¸ÑÐ¾Ð²Ð°Ð½Ð½ÑÑ Ð¸Ð· ÑÐ°Ð·Ð»Ð¸ÑÐ½ÑÑ Ð¸ÑÑÐ¾ÑÐ½Ð¸ÐºÐ¾Ð². Ð ÑÐ°Ðº Ð¶Ðµ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° ETL ÐºÐ¾Ð½Ð²ÐµÐ¹ÐµÑÐ° Ð´Ð»Ñ ÑÐ±Ð¾ÑÐ° Ð¸ Ð¿ÐµÑÐ¸Ð¾Ð´Ð¸ÑÐµÑÐºÐ¾Ð³Ð¾ Ð¾Ð±Ð½Ð¾Ð²Ð»ÐµÐ½Ð¸Ñ ÑÑÐ¸Ñ Ð´Ð°Ð½Ð½ÑÑ. Ð¡Ð¸ÑÑÐµÐ¼Ð° Ð¿ÑÐµÐ´Ð¾ÑÑÐ°Ð²Ð»ÑÐµÑ ÑÑÐ½ÐºÑÐ¸Ð¸ Ð´Ð»Ñ Ð´Ð¾ÑÑÐ¸Ð¶ÐµÐ½Ð¸Ñ ÑÐ»ÐµÐ´ÑÑÑÐ¸Ñ ÑÐµÐ»ÐµÐ¹: ÑÐ¾ÐºÑÐ°ÑÐµÐ½Ð¸Ðµ Ð²ÑÐµÐ¼ÐµÐ½Ð¸ Ð¸ ÑÐ°ÑÑÐ¾Ð´Ð¾Ð² Ð½Ð° Ð¸Ð·ÑÑÐµÐ½Ð¸Ðµ ÑÑÐ½ÐºÐ° Ð¸ Ð¿Ð¾Ð¸ÑÐº ÐºÐ¾Ð¼Ð¿Ð°Ð½Ð¸Ð¹ Ñ ÑÐ¾Ð±ÑÑÐ²ÐµÐ½Ð½ÑÐ¼Ð¸ ÑÐµÑÐ½Ð¾Ð»Ð¾Ð³Ð¸ÑÐµÑÐºÐ¸Ð¼Ð¸ ÑÐµÑÐµÐ½Ð¸ÑÐ¼Ð¸ Ð´Ð»Ñ Ð¿Ð¾Ð²ÑÑÐµÐ½Ð¸Ñ Ð²ÑÑÑÑÐºÐ¸ Ð¸Ð»Ð¸ ÑÐ¾ÐºÑÐ°ÑÐµÐ½Ð¸Ñ Ð¸Ð·Ð´ÐµÑÐ¶ÐµÐº Ð·Ð° ÑÑÐµÑ Ð½Ð¾Ð²ÑÑ ÑÐµÑÐ½Ð¾Ð»Ð¾Ð³Ð¸Ð¹. ÐÑÐ» Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½ Ð°Ð½Ð°Ð»Ð¸Ð· Ð¸ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ Ð½ÐµÑÐºÐ¾Ð»ÑÐºÐ¸Ñ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð¾Ð² Ðº ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸ ÑÐ¸ÑÑÐµÐ¼Ñ Ð¿Ð¾Ð¸ÑÐºÐ° Ð¿Ð¾ ÑÐµÐµÑÑÑÐ°Ð¼ Ð´Ð°Ð½Ð½ÑÑ Ñ ÑÑÐµÑÐ¾Ð¼ ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¾Ð³Ð¾ Ð·Ð°Ð´Ð°Ð½Ð¸Ñ. ÐÑÐ¾Ð²ÐµÐ´ÐµÐ½ Ð´ÐµÑÐ°Ð»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· Ð²Ð¾Ð·Ð¼Ð¾Ð¶Ð½Ð¾ÑÑÐµÐ¹ Elasticsearch Ð¸ PostgreSQL Ð´Ð»Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð¿Ð¾ÑÑÐ°Ð²Ð»ÐµÐ½Ð½ÑÑ ÑÑÐµÐ±Ð¾Ð²Ð°Ð½Ð¸Ð¹. Ð ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐµ Ð±ÑÐ»Ð° ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð° Ð¿Ð¾Ð¸ÑÐºÐ¾Ð²Ð°Ñ ÑÐ¸ÑÑÐµÐ¼Ð° Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ Ð¿Ð¾Ð¸ÑÐºÐ¾Ð²Ð¾Ð³Ð¾ Ð´Ð²Ð¸Ð¶ÐºÐ° Elasticsearch. Ð Ð´Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÐµ Ð¾Ð¿Ð¸ÑÐ°Ð½Ð° Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÐ° ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð½Ð¾Ð¹ ÑÐ¸ÑÑÐµÐ¼Ñ, ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ ÑÐ°Ð·Ð»Ð¸ÑÐ½ÑÑ Ð¿Ð¾Ð¸ÑÐºÐ¾Ð²ÑÑ Ð´Ð²Ð¸Ð¶ÐºÐ¾Ð² Ð¸ Ð°Ð½Ð°Ð»Ð¸Ð· Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½ÑÑ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð²: Ð²ÑÐµÐ¼Ñ ÑÐ°Ð±Ð¾ÑÑ Ð¿Ð¾Ð¸ÑÐºÐ°, ÑÐ¾ÑÐ½Ð¾ÑÑÑ Ð²ÑÐ´Ð°ÑÐ¸ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð² Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð½Ð¾Ð¹ Ð¼ÐµÑÐ¾Ð´Ð¸ÐºÐ¸ Ð¿ÑÐ¾Ð²ÐµÑÐºÐ¸. ÐÐ¾ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ°Ð¼ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ»Ð° ÑÑÑÐ°Ð½Ð¾Ð²Ð»ÐµÐ½Ð° ÑÑÐ¾Ð¶ÐµÑÑÑ Ð¼ÐµÑÑÐ¸Ðº ÑÐ¾ÑÐ½Ð¾ÑÑÐ¸ Ð¸ ÑÐºÐ¾ÑÐ¾ÑÑÐ¸ Ð¿Ð¾Ð¸ÑÐºÐ¾Ð²ÑÑ Ð´Ð²Ð¸Ð¶ÐºÐ¾Ð² Elasticsearch Ð¸ PostgreSQL Ð¿ÑÐ¸ Ð¿Ð¾Ð¸ÑÐºÐµ Ñ Ð±Ð°Ð·Ð¾Ð²Ð¾Ð¹ ÑÑÐ½ÐºÑÐ¸Ð¾Ð½Ð°Ð»ÑÐ½Ð¾ÑÑÑÑ. ÐÐ»Ñ Ð¿Ð¾Ð¸ÑÐºÐ° Ð¿Ð¾ Ð±Ð¾Ð»ÑÑÐ¸Ð¼ Ð¾Ð±ÑÐµÐ¼Ð°Ð¼ Ð´Ð°Ð½Ð½ÑÑ Ñ ÑÐ»Ð¾Ð¶Ð½ÑÐ¼Ð¸ Ð¿Ð°ÑÐ°Ð¼ÐµÑÑÐ°Ð¼Ð¸ ÑÐ°Ð½Ð¶Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Ð±Ð¾Ð»ÐµÐµ ÑÑÑÐµÐºÑÐ¸Ð²Ð½ÑÐ¼ Ð¾ÐºÐ°Ð·Ð°Ð»ÑÑ Elasticsearch.
description: This work is devoted to the study of approaches to the construction of a full-text search on a limited set of data. The purpose of this work is to develop a full-text search system for data registries of Russian companies aggregated from various sources. As well as the development of an ETL pipeline for collecting and periodically updating this data. The system provides functions to achieve the following goals: reduce the time and cost of studying the market and finding companies with their own technological solutions to increase revenue or reduce costs through new technologies. An analysis and comparison of several approaches to the implementation of a search system in data registries was carried out, taking into account the terms of reference. A detailed analysis of the capabilities of Elasticsearch and PostgreSQL was carried out to implement the requirements. As a result, a search engine was developed based on the Elasticsearch search engine. This paper describes the architecture of the developed system, a comparison of various search engines and an analysis of the results obtained: the search time, the accuracy of the results. Based on the results of the work carried out, the similarity of the precision and speed metrics of Elasticsearch and PostgreSQL when searching with basic functionality was established. For searching large amounts of data with exceptional ranking parameters, Elasticsearch is more efficient.