title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¼ÐµÑÐ¾Ð´Ð° Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ Ð½ÐµÑÑÐ°ÑÐ½ÑÑ ÑÐ¸ÑÑÐ°ÑÐ¸Ð¹ Ð² Ð²Ð¸Ð´ÐµÐ¾Ð¿Ð¾ÑÐ¾ÐºÐµ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð²Ð¸Ð·ÑÐ°Ð»ÑÐ½Ð¾-ÑÐ·ÑÐºÐ¾Ð²Ð¾Ð¹ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½Ð¾Ð¹ ÑÐµÑÐ¸
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Development of a method for detecting abnormal situations in a video stream using a visual language neural network
Creators: name: Ð§Ð°Ð½ÑÐ¸ÐºÐ¾Ð², ÐÐ¼Ð¸ÑÑÐ¸Ð¹
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2024
Subjects: Ð²Ð¸Ð´ÐµÐ¾Ð°Ð½Ð°Ð»Ð¸ÑÐ¸ÐºÐ°
Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ðµ Ð½ÐµÑÑÐ°ÑÐ½ÑÑ ÑÐ¸ÑÑÐ°ÑÐ¸Ð¹
Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸
Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
Ð²Ð¸Ð·ÑÐ°Ð»ÑÐ½Ð¾-ÑÐ·ÑÐºÐ¾Ð²ÑÐµ Ð¼Ð¾Ð´ÐµÐ»Ð¸
Ð¾Ð¿Ð¸ÑÐ°Ð½Ð¸Ðµ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹ Ð½Ð° ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð¼ ÑÐ·ÑÐºÐµ
video analytics
emergency situation detection
neural networks
deep learning
visual-language models
description of images in natural language

description: ÐÐ±ÑÐµÐºÑÐ¾Ð¼ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°Ð½Ð¸Ñ ÑÐ²Ð»ÑÑÑÑÑ Ð²Ð¸Ð·ÑÐ°Ð»ÑÐ½Ð¾-ÑÐ·ÑÐºÐ¾Ð²ÑÐµ Ð³Ð»ÑÐ±Ð¾ÐºÐ¸Ðµ Ð½ÐµÐ¹ÑÐ¾Ð½-Ð½ÑÐµ ÑÐµÑÐ¸, ÑÐ¾ÑÐ¼Ð¸ÑÑÑÑÐ¸Ðµ Ð¾Ð¿Ð¸ÑÐ°Ð½Ð¸Ñ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ð¹ Ð½Ð° Â«ÐµÑÑÐµÑÑÐ²ÐµÐ½Ð½Ð¾Ð¼Â» ÑÐ·ÑÐºÐµ. Ð¦ÐµÐ»Ñ Ð´Ð°Ð½Ð½Ð¾Ð¹ ÑÐ°Ð±Ð¾ÑÑ Ð·Ð°ÐºÐ»ÑÑÐ°ÐµÑÑÑ Ð² ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐµ Ð¼ÐµÑÐ¾Ð´Ð° Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ Ð½ÐµÑÑÐ°ÑÐ½ÑÑ ÑÐ¸ÑÑÐ°ÑÐ¸Ð¹ Ð² Ð²Ð¸Ð´ÐµÐ¾Ð¿Ð¾ÑÐ¾ÐºÐµ Ñ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸ÐµÐ¼ Ð²Ð¸Ð·ÑÐ°Ð»ÑÐ½Ð¾-ÑÐ·ÑÐºÐ¾Ð²Ð¾Ð¹ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½Ð¾Ð¹ ÑÐµÑÐ¸. Ð ÑÐ°Ð±Ð¾ÑÐµ Ð¿ÑÐ¾Ð²ÐµÐ´ÐµÐ½ Ð°Ð½Ð°Ð»Ð¸Ð· ÑÑÑÐµÑÑÐ²ÑÑÑÐ¸Ñ Ð¼ÐµÑÐ¾Ð´Ð¾Ð², Ð¿Ð¾ÐºÐ°Ð·Ð°Ð²ÑÐ¸Ð¹, ÑÑÐ¾ Ð²Ð¸Ð·Ñ-Ð°Ð»ÑÐ½Ð¾-ÑÐ·ÑÐºÐ¾Ð²ÑÐµ Ð½ÐµÐ¹ÑÐ¾ÑÐµÑÐµÐ²ÑÐµ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¾Ð±Ð»Ð°Ð´Ð°ÑÑ Ð·Ð½Ð°ÑÐ¸ÑÐµÐ»ÑÐ½ÑÐ¼Ð¸ Ð¿ÑÐµÐ¸Ð¼ÑÑÐµÑÑÐ²Ð°-Ð¼Ð¸ Ð±Ð»Ð°Ð³Ð¾Ð´Ð°ÑÑ ÑÐ¿Ð¾ÑÐ¾Ð±Ð½Ð¾ÑÑÐ¸ ÑÐ¾Ð¿Ð¾ÑÑÐ°Ð²Ð»ÑÑÑ Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ñ Ð¸ ÑÐµÐºÑÑ Ð±ÐµÐ· Ð´Ð¾Ð¿Ð¾Ð»Ð½Ð¸ÑÐµÐ»ÑÐ½Ð¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ. Ð¡ÑÐµÐ´Ð¸ Ð½Ð¸Ñ Ð´Ð»Ñ Ð´Ð°Ð»ÑÐ½ÐµÐ¹ÑÐµÐ³Ð¾ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ð¸Ñ Ð²ÑÐ±ÑÐ°Ð½Ð° Ð¼Ð¾Ð´ÐµÐ»Ñ CLIP, Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ ÐºÐ¾ÑÐ¾ÑÐ¾Ð¹ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ð¾ ÑÑÐ¸ Ð²Ð°ÑÐ¸Ð°ÑÐ¸Ð¸ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ Ð½Ðµ-ÑÑÐ°ÑÐ½ÑÑ ÑÐ¸ÑÑÐ°ÑÐ¸Ð¹ Ð² Ð²Ð¸Ð´ÐµÐ¾Ð¿Ð¾ÑÐ¾ÐºÐµ. ÐÐ½Ð¸ ÑÐµÐ°Ð»Ð¸Ð·Ð¾Ð²Ð°Ð½Ñ Ð² Ð²Ð¸Ð´Ðµ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ñ Ð½Ð° ÑÐ·ÑÐºÐµ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð¸ÑÐ¾Ð²Ð°Ð½Ð¸Ñ Python. ÐÑÐ¾Ð¼Ðµ ÑÐ¾Ð³Ð¾, ÑÐ¾Ð±ÑÐ°Ð½ Ð¸ Ð°Ð½Ð½Ð¾ÑÐ¸ÑÐ¾Ð²Ð°Ð½ Ð½Ð°Ð±Ð¾Ñ Ð²Ð¸Ð·ÑÐ°Ð»ÑÐ½ÑÑ Ð¿ÑÐ¸Ð¼ÐµÑÐ¾Ð², Ð¸ ÑÑÐ¾ÑÐ¼Ð¸ÑÐ¾Ð²Ð°Ð½ Ð½Ð°Ð±Ð¾Ñ ÑÐµÐºÑÑÐ¾Ð²ÑÑ Ð¾Ð¿Ð¸ÑÐ°Ð½Ð¸Ð¹ Ð½ÐµÑÑÐ°ÑÐ½ÑÑ ÑÐ¸ÑÑÐ°ÑÐ¸Ð¹. ÐÐ¾ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ°Ð¼ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½ÑÐ¾Ð² Ð²Ð°ÑÐ¸Ð°Ð½Ñ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð° Ñ Ð¾Ð±ÑÑÐ°ÐµÐ¼ÑÐ¼ Ð»Ð¸Ð½ÐµÐ¹Ð½ÑÐ¼ ÐºÐ»Ð°ÑÑÐ¸ÑÐ¸ÐºÐ°ÑÐ¾ÑÐ¾Ð¼, Ð¸ÑÐ¿Ð¾Ð»ÑÐ·ÑÑÑÐ¸Ð¼ ÑÐµÐºÑÑÐ¾Ð²ÑÐµ Ð¾Ð¿Ð¸ÑÐ°Ð½Ð¸Ñ Ð¸ Ð²Ð¸Ð·ÑÐ°Ð»ÑÐ½ÑÐµ Ð¿ÑÐ¸Ð¼ÐµÑÑ, Ð¿ÑÐ¾Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ¸ÑÐ¾Ð²Ð°Ð» Ð½Ð°Ð¸Ð»ÑÑÑÐ¸Ðµ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÑ Ð¿Ð¾ Ð¼ÐµÑÑÐ¸ÐºÐµ F-score (64%), Ð° ÑÐ°ÐºÐ¶Ðµ Ð¼Ð°Ð»Ð¾Ðµ Ð²ÑÐµÐ¼Ñ ÑÐµÐ°ÐºÑÐ¸Ð¸ (0,66 Ñ) Ð¸ Ð²ÑÑÐ¾ÐºÑÑ Ð´Ð¾Ð»Ñ Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð½ÑÑ Ð¸Ð½ÑÐµÑÐ²Ð°Ð»Ð¾Ð² (96%). Ð Ð´Ð¾Ð¿Ð¾Ð»Ð½ÐµÐ½Ð¸Ðµ Ðº ÑÑÐ¾Ð¼Ñ, ÑÑÑÐ°Ð½Ð¾Ð²Ð»ÐµÐ½Ð¾, ÑÑÐ¾ Ð´Ð»Ñ Ð¾Ð±Ð½Ð°ÑÑÐ¶ÐµÐ½Ð¸Ñ Ð½ÐµÐºÐ¾ÑÐ¾ÑÑÑ ÐºÐ»Ð°ÑÑÐ¾Ð² Ð½ÐµÑÑÐ°ÑÐ½ÑÑ ÑÐ¸ÑÑÐ°ÑÐ¸Ð¹ Ð² Ð´Ð°Ð»ÑÐ½ÐµÐ¹ÑÐµÐ¼ Ð½ÐµÐ¾Ð±ÑÐ¾Ð´Ð¸Ð¼Ð¾ ÑÑÐ¸ÑÑÐ²Ð°ÑÑ Ð²ÑÐµÐ¼ÐµÐ½Ð½ÑÑ Ð¿Ð¾ÑÐ»ÐµÐ´Ð¾Ð²Ð°ÑÐµÐ»ÑÐ½Ð¾ÑÑÑ ÐºÐ°Ð´ÑÐ¾Ð² Ð²Ð¸Ð´ÐµÐ¾Ð¿Ð¾ÑÐ¾ÐºÐ°.
description: The object of this research is visual-language deep neural networks that generate descriptions of images in natural language. The aim of this work is to develop a method for detecting anomalous situations in video streams using a visual-language neural network. The study includes an analysis of existing methods, revealing that visual-language neural network models have significant advantages due to their ability to associate images with text without additional training. Among these models, CLIP was selected for further use, and three variations of the algorithm for detecting anomalous situations in video streams were developed based on it. These algorithms were implemented in a Python program. Additionally, a dataset of visual examples was collected and annotated, and a set of textual descriptions of anomalous situations was compiled. Experimental results showed that the variant of the algorithm with a trainable linear classifier using textual descriptions and visual examples demonstrated the best performance in terms of the F-score metric (64%), as well as a short response time (0.66 s) and a high detection rate of intervals (96%). Furthermore, it was established that for detecting certain classes of anomalous situations, it is necessary to consider the temporal sequence of video frames in future work.