title: Ð¡ÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ Ð¸ ÑÐ»ÑÑÑÐµÐ½Ð¸Ðµ ÐºÐ°ÑÐµÑÑÐ²Ð° ÑÐ°Ð±Ð¾ÑÑ open source Ð°ÑÑ Ð¸ÑÐµÐºÑÑÑ Ð´Ð»Ñ Ð·Ð°Ð´Ð°ÑÐ¸ NER Ð² ÑÑÐ»Ð¾Ð²Ð¸ÑÑ Ð¾Ð³ÑÐ°Ð½Ð¸ÑÐµÐ½Ð½Ð¾ÑÑÐ¸ ÑÐµÑÑÑÑÐ¾Ð²
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Comparison and improvementof the performance of open-source architectures for the NER task under resourceconstraints
Creators: name: ÐÐ²Ð¾Ð·Ð´ÐµÐ², Ð¡Ð²ÑÑÐ¾ÑÐ»Ð°Ð²
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2024
Subjects: ÑÐ¼Ð±ÐµÐ´Ð´Ð¸Ð½Ð³
LSTM
transformers
ÑÐ¾ÐºÐµÐ½
LLM
self-attention
embedding
token

description: Ð¦ÐµÐ»ÑÑ ÑÐ°Ð±Ð¾ÑÑ ÑÐ²Ð»ÑÐµÑÑÑ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ Ð¸ Ð¿Ð¾Ð´Ð±Ð¾Ñ Ð¸Ð½ÑÑÑÑÐ¼ÐµÐ½ÑÐ¾Ð² Ð´Ð»Ñ ÑÐ¾Ð·Ð´Ð°Ð½Ð¸Ñ ÑÐµÑÐ²Ð¸ÑÐ° Ð¿Ð¾ Ð¸Ð·Ð²Ð»ÐµÑÐµÐ½Ð¸Ñ ÑÑÑÐ½Ð¾ÑÑÐµÐ¹ Ð¸Ð· ÐºÐ¾ÑÐ¿Ð¾ÑÐ°ÑÐ¸Ð²Ð½ÑÑ Ð´Ð¾ÐºÑÐ¼ÐµÐ½ÑÐ¾Ð². ÐÑÐµÐ´Ð¼ÐµÑÐ¾Ð¼ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ ÑÐ²Ð»ÑÑÑÑÑ ÑÐ¾Ð²ÑÐµÐ¼ÐµÐ½Ð½ÑÐµ Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÑ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹, Ð¼ÐµÑÐ¾Ð´Ñ Ð¸Ñ Ð´Ð¾Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð¸ ÑÐ°Ð·Ð²ÑÑÑÑÐ²Ð°Ð½Ð¸Ñ. ÐÐ°Ð´Ð°ÑÐ¸, ÑÐµÑÐ°ÐµÐ¼ÑÐµ Ð² ÑÐ¾Ð´Ðµ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ: 1. Ð Ð°Ð·Ð¼ÐµÑÐºÐ° Ð´Ð°Ð½Ð½ÑÑ Ð´Ð»Ñ Ð¾ÑÐµÐ½ÐºÐ¸ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð¸ Ð´Ð¾Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ. 2. Ð ÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ð²ÑÐ´ÐµÐ»ÐµÐ½Ð¸Ñ ÑÐµÐ»ÐµÐ²Ð°Ð½ÑÐ½ÑÑ ÑÐ°ÑÑÐµÐ¹ Ð´Ð¾ÐºÑÐ¼ÐµÐ½ÑÐ¾Ð² Ð´Ð»Ñ Ð±Ð¾ÑÐ±Ñ Ñ Ð¾Ð³ÑÐ°Ð½Ð¸ÑÐµÐ½Ð½ÑÐ¼ ÐºÐ¾Ð½ÑÐµÐºÑÑÐ¾Ð¼ ÑÐ·ÑÐºÐ¾Ð²ÑÑ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹. 3. ÐÐ¾Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ ÑÐ·ÑÐºÐ¾Ð²ÑÑ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹ Ð¸ Ð´Ð¾ÑÑÐ¸Ð¶ÐµÐ½Ð¸Ðµ Ð¾Ð¿ÑÐ¸Ð¼Ð°Ð»ÑÐ½Ð¾Ð¹ ÑÐºÐ¾ÑÐ¾ÑÑÐ¸ Ð¸Ñ Ð¾ÑÐºÐ»Ð¸ÐºÐ°. Ð ÑÐ¾Ð´Ðµ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ»Ð° Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð° Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÐ° transformers Ð¸ Ð¿Ð¾Ð´ÑÐ¾Ð±Ð½Ð° ÑÐ°Ð·Ð¾Ð±ÑÐ°Ð½ Ð¿ÑÐ¸Ð½ÑÐ¸Ð¿ Ð´ÐµÐ¹ÑÑÐ²Ð¸Ñ ÑÐ»Ð¾Ñ self-attention. ÐÑÐ»Ð¸ Ð¿ÑÐ¾Ð°Ð½Ð°Ð»Ð¸Ð·Ð¸ÑÐ¾Ð²Ð°Ð½Ñ ÑÐ¿Ð¾ÑÐ¾Ð±Ñ ÑÐ»ÑÑÑÐµÐ½Ð¸Ñ ÐºÐ°ÑÐµÑÑÐ²Ð° Ð¸ ÑÐ²ÐµÐ»ÐµÑÐµÐ½Ð¸Ñ ÑÐºÐ¾ÑÐ¾ÑÑÐ¸ Ð³ÐµÐ½ÐµÑÐ°ÑÐ¸Ð¸ Ð¾ÑÐ²ÐµÑÐ¾Ð² ÑÐ·ÑÐºÐ¾Ð²ÑÑ Ð¼Ð¾Ð´ÐµÐ»ÐµÐ¹. Ð ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐµ ÑÐ°Ð±Ð¾ÑÑ Ð±ÑÐ» ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½ ÑÐµÑÐ²Ð¸Ñ, ÐºÐ¾ÑÐ¾ÑÑÐ¹ Ð¿Ð¾Ð·Ð²Ð¾Ð»ÑÐµÑ Ð¸Ð·Ð²Ð»ÐµÐºÐ°ÑÑ Ð½ÐµÐ¾Ð±ÑÐ¾Ð´Ð¸Ð¼ÑÐµ Ð´Ð°Ð½Ð½ÑÐµ Ð¸Ð· Ð´Ð¾ÐºÑÐ¼ÐµÐ½ÑÐ¾Ð², Ð´Ð»Ñ Ð¾Ð¿ÑÐ¸Ð¼Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð¿ÑÐ¾ÑÐµÑÑÐ¾Ð² Ð²Ð½ÑÑÑÐ¸ ÐºÐ¾Ð¼Ð¿Ð°Ð½Ð¸Ð¸.
description: The objective of this work is to compare and select tools for creating a service to extract entities from corporate documents. The subject of the study includes modern neural network architectures, methods of their fine-tuning, and deployment. The tasks addressed in the course of the research are: 1. Data annotation for quality assessment and fine-tuning. 2. Implementation of an algorithm to extract relevant parts of documents to address the limited context of language models. 3. Fine-tuning language models and achieving optimal response speed. During the work, the transformers architecture was studied, and the functioning of the self-attention layer was thoroughly analyzed. Methods to improve the quality and increase the response speed of language models were examined. As a result of the work, a service was developed that allows the extraction of necessary data from documents to optimize processes within the company.