title: Ð Ð°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° ÑÐ¸ÑÑÐµÐ¼Ñ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¶ÐµÑÑÐ¾Ð² Ð¶ÐµÑÑÐ¾Ð²Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ° Ð¿Ð¾ Ð²Ð¸Ð´ÐµÐ¾Ð¸Ð·Ð¾Ð±ÑÐ°Ð¶ÐµÐ½Ð¸Ñ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Development of a system for recognizing sign language gestures by video
Creators: name: ÐÐ¾Ð»Ð¾Ð¼ÐµÐµÑ, ÐÐ°ÑÐ¸Ñ
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2020
Subjects: Ð°Ð¼ÐµÑÐ¸ÐºÐ°Ð½ÑÐºÐ¸Ð¹ Ð¶ÐµÑÑÐ¾Ð²Ð¾Ð¹ ÑÐ·ÑÐº
Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
ÑÐ²ÐµÑÑÐ¾ÑÐ½ÑÐµ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÐµ ÑÐµÑÐ¸
Ð¿Ð»Ð¾ÑÐ½ÑÐ¹ Ð¾Ð¿ÑÐ¸ÑÐµÑÐºÐ¸Ð¹ Ð¿Ð¾ÑÐ¾Ðº
american sign language
machine learning
deep learning
convolutional neural networks
dense optical flow

description: ÐÐ°Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð¿Ð¾ÑÐ²ÑÑÐµÐ½Ð° ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐµ ÑÐ¸ÑÑÐµÐ¼Ñ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¿Ð¾Ð´Ð¼Ð½Ð¾Ð¶ÐµÑÑÐ²Ð° ÑÑÐ°ÑÐ¸ÑÐµÑÐºÐ¸Ñ Ð¸ Ð´Ð¸Ð½Ð°Ð¼Ð¸ÑÐµÑÐºÐ¸Ñ Ð¶ÐµÑÑÐ¾Ð² Ð°Ð¼ÐµÑÐ¸ÐºÐ°Ð½ÑÐºÐ¾Ð³Ð¾ Ð¶ÐµÑÑÐ¾Ð²Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°. ÐÐ»Ñ ÑÐµÑÐµÐ½Ð¸Ñ ÑÑÐ¾Ð¹ Ð·Ð°Ð´Ð°ÑÐ¸ Ð¿ÑÐµÐ´Ð»Ð°Ð³Ð°ÐµÑÑÑ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°ÑÑ Ð¼Ð¾Ð´ÐµÐ»Ñ Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ ÑÐ²ÐµÑÑÐ¾ÑÐ½ÑÑ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½ÑÑ ÑÐµÑÐµÐ¹. ÐÐ»Ñ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·ÑÑÑÑÑ Ð½Ð°Ð±Ð¾ÑÑ Ð´Ð°Ð½Ð½ÑÑ NCSLGR [1] Ð¸ WLASL [2] Ð´Ð»Ñ Ð´Ð¸Ð½Ð°Ð¼Ð¸ÑÐµÑÐºÐ¸Ñ Ð¸ ÑÑÐ°ÑÐ¸ÑÐµÑÐºÐ¸Ñ Ð¶ÐµÑÑÐ¾Ð² ÑÐ¾Ð¾ÑÐ²ÐµÑÑÑÐ²ÐµÐ½Ð½Ð¾. ÐÐ»Ñ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¶ÐµÑÑÐ¾Ð² Ð²ÑÐ¾Ð´Ð½Ð¾Ð¹ Ð¿Ð¾ÑÐ¾Ðº ÐºÐ°Ð´ÑÐ¾Ð² Ð±ÑÐ» Ð¿ÑÐµÐ¾Ð±ÑÐ°Ð·Ð¾Ð²Ð°Ð½ Ð² Ð¾Ð¿ÑÐ¸ÑÐµÑÐºÐ¸Ð¹ Ð¿Ð¾ÑÐ¾Ðº Ñ ÑÐ²ÐµÑÐ¾Ð²Ð¾Ð¹ Ð¼Ð¾Ð´ÐµÐ»ÑÑ. ÐÐ»Ñ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ð¸ Ð²ÑÐµÑ Ð½ÐµÐ¾Ð±ÑÐ¾Ð´Ð¸Ð¼ÑÑ Ð¼ÐµÑÐ¾Ð´Ð¾Ð² Ð¸ Ð°Ð»Ð³Ð¾ÑÐ¸ÑÐ¼Ð¾Ð² Ð±ÑÐ»Ð¸ Ð¸ÑÐ¿Ð¾Ð»ÑÐ·Ð¾Ð²Ð°Ð½Ñ Ð±Ð¸Ð±Ð»Ð¸Ð¾ÑÐµÐºÐ¸ TensorFlow, Keras, OpenCV, scikit-learn.
Ð­ÑÐ° ÑÐ°Ð±Ð¾ÑÐ° Ð¾ÑÐ½Ð¾ÑÐ¸ÑÑÑ Ðº ÑÑÐµÑÐµ Ð³Ð»ÑÐ±Ð¾ÐºÐ¾Ð³Ð¾ Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ð¸ ÐºÐ¾Ð¼Ð¿ÑÑÑÐµÑÐ½Ð¾Ð³Ð¾ Ð·ÑÐµÐ½Ð¸Ñ. Ð Ð¿ÐµÑÐ²Ð¾Ð¹ Ð³Ð»Ð°Ð²Ðµ Ð¿ÑÐ¸Ð²Ð¾Ð´Ð¸ÑÑÑ ÑÑÐ°Ð²Ð½Ð¸ÑÐµÐ»ÑÐ½ÑÐ¹ Ð°Ð½Ð°Ð»Ð¸Ð· ÑÑÑÐµÑÑÐ²ÑÑÑÐ¸Ñ ÑÐ¸ÑÑÐµÐ¼ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¶ÐµÑÑÐ¾Ð² Ð¶ÐµÑÑÐ¾Ð²Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°, Ð° ÑÐ°ÐºÐ¶Ðµ Ð°Ð½Ð°Ð»Ð¸Ð· Ð¾ÑÐ½Ð¾Ð²Ð½ÑÑ Ð½ÐµÐ´Ð¾ÑÑÐ°ÑÐºÐ¾Ð² ÑÑÑÐµÑÑÐ²ÑÑÑÐ¸Ñ ÑÐµÑÐµÐ½Ð¸Ð¹. ÐÑÐ»Ð¸ ÑÑÐ¾ÑÐ¼ÑÐ»Ð¸ÑÐ¾Ð²Ð°Ð½Ñ Ð¾ÑÐ½Ð¾Ð²Ð½ÑÐµ ÑÑÐµÐ±Ð¾Ð²Ð°Ð½Ð¸Ñ Ðº ÑÐ°Ð·ÑÐ°Ð±Ð°ÑÑÐ²Ð°ÐµÐ¼Ð¾Ð¹ ÑÐ¸ÑÑÐµÐ¼Ðµ.
ÐÐ¾ Ð²ÑÐ¾ÑÐ¾Ð¹ Ð³Ð»Ð°Ð²Ðµ Ð¾Ð¿Ð¸ÑÑÐ²Ð°ÑÑÑÑ Ð¾ÑÐ½Ð¾Ð²Ð½ÑÐµ ÑÐµÐ¾ÑÐµÑÐ¸ÑÐµÑÐºÐ¸Ðµ Ð·Ð½Ð°Ð½Ð¸Ñ, ÐºÐ¾ÑÐ¾ÑÑÐµ Ð½ÐµÐ¾Ð±ÑÐ¾Ð´Ð¸Ð¼Ñ Ð´Ð»Ñ ÑÑÐ¿ÐµÑÐ½Ð¾Ð³Ð¾ Ð½Ð°Ð¿Ð¸ÑÐ°Ð½Ð¸Ñ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ñ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¶ÐµÑÑÐ¾Ð².
Ð ÑÑÐµÑÑÐµÐ¹ Ð³Ð»Ð°Ð²Ðµ Ð¾Ð¿Ð¸ÑÑÐ²Ð°ÐµÑÑÑ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° ÑÐµÑÐµÐ½Ð¸Ñ Ð´Ð»Ñ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ Ð¶ÐµÑÑÐ¾Ð² Ð°Ð¼ÐµÑÐ¸ÐºÐ°Ð½ÑÐºÐ¾Ð³Ð¾ Ð¶ÐµÑÑÐ¾Ð²Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°, Ð°ÑÑÐ¸ÑÐµÐºÑÑÑÐ° Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ñ Ð¸ ÐµÐµ ÑÐµÐ°Ð»Ð¸Ð·Ð°ÑÐ¸Ñ. 
Ð ÑÐµÑÐ²ÐµÑÑÐ¾Ð¹ Ð³Ð»Ð°Ð²Ðµ Ð¿ÑÐ¸Ð²ÐµÐ´ÐµÐ½Ñ ÑÐºÑÐ¿ÐµÑÐ¸Ð¼ÐµÐ½ÑÑ, Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ¸ÑÑÑÑÐ¸Ðµ ÑÐ¾ÑÐ½Ð¾ÑÑÑ Ð¿ÑÐ¸Ð²ÐµÐ´ÐµÐ½Ð½Ð¾Ð³Ð¾ ÑÐµÑÐµÐ½Ð¸Ñ, Ð¸ Ð´ÐµÐ¼Ð¾Ð½ÑÑÑÐ°ÑÐ¸Ñ ÑÐ°Ð±Ð¾ÑÑ Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð½Ð¾Ð³Ð¾ ÑÐµÑÐµÐ½Ð¸Ñ. 
Ð ÐµÐ·ÑÐ»ÑÑÐ°Ñ ÑÐ°Ð±Ð¾ÑÑ â Ð¿ÑÐ¾Ð³ÑÐ°Ð¼Ð¼Ð°, ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°ÑÑÐ°Ñ Ð¿Ð¾Ð´Ð¼Ð½Ð¾Ð¶ÐµÑÑÐ²Ð¾ Ð´Ð¸Ð½Ð°Ð¼Ð¸ÑÐµÑÐºÐ¸Ñ Ð¸ ÑÑÐ°ÑÐ¸ÑÐµÑÐºÐ¸Ñ Ð¶ÐµÑÑÐ¾Ð² Ð°Ð¼ÐµÑÐ¸ÐºÐ°Ð½ÑÐºÐ¾Ð³Ð¾ Ð¶ÐµÑÑÐ¾Ð²Ð¾Ð³Ð¾ ÑÐ·ÑÐºÐ°. Ð¢Ð¾ÑÐ½Ð¾ÑÑÑ ÑÐ°ÑÐ¿Ð¾Ð·Ð½Ð°Ð²Ð°Ð½Ð¸Ñ 97%.
description: The primary objective of this work is to create a recognition system for a subset of static and dynamic gestures of the American sign language. To solve this problem, I proposed to use a model based on convolutional neural networks. The experimental results carried on two well-known datasets â NCSLGR Corpus [1] and WLASL [2]. As a pre-processing step the input stream of the frames was converted into a color-coded optical flow. To implement all the methods and algorithms TensorFlow, Keras, OpenCV, scikit-learn libraries were used.
This work belongs to the field of deep learning and computer vision. The first chapter provides a related work, comparative analysis of existing sign language recognition systems, as well as an analysis of the main disadvantages of existing solutions. Also, requirements were formulated in this chapter.
The second chapter describes the basic theoretical knowledge that is needed to successfully develop a gesture recognition system.
The third chapter describes the development of a solution for recognizing American sign language, the architecture of the program and its implementation.
The fourth chapter contains experiments demonstrating the accuracy of the solution and demonstrates how the solution works.
The result of the work is a program that recognizes a subset of the dynamic and static gestures of the American sign language. Recognition accuracy is 97%.