title: ÐÐ±ÑÑÐµÐ½Ð¸Ðµ Ñ Ð¿Ð¾Ð´ÐºÑÐµÐ¿Ð»ÐµÐ½Ð¸ÐµÐ¼ Ð½ÐµÐ¹ÑÐ¾Ð½Ð½Ð¾Ð¹ ÑÐµÑÐ¸, Ð¾ÑÐ½Ð¾Ð²Ð°Ð½Ð½Ð¾Ðµ Ð½Ð° Ð»ÑÐ±Ð¾Ð¿ÑÑÑÑÐ²Ðµ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ ÑÐ°Ð¼Ð¾ÐºÐ¾Ð½ÑÑÐ¾Ð»Ð¸ÑÑÐµÐ¼Ð¾Ð³Ð¾ Ð¿ÑÐ¾Ð³Ð½Ð¾Ð·Ð° Ð² Action Ð¸Ð³ÑÐµ
title: Ð²ÑÐ¿ÑÑÐºÐ½Ð°Ñ ÐºÐ²Ð°Ð»Ð¸ÑÐ¸ÐºÐ°ÑÐ¸Ð¾Ð½Ð½Ð°Ñ ÑÐ°Ð±Ð¾ÑÐ° Ð±Ð°ÐºÐ°Ð»Ð°Ð²ÑÐ°
titleType: Subtitle
title: Neural network reinforcement learning, based on curiosity using self-supervised prediction in an action game
Creators: name: ÐÑÐºÑÐ¾Ð², ÐÑÐ¸Ð³Ð¾ÑÐ¸Ð¹
Publisher: name: Ð¡Ð°Ð½ÐºÑ-ÐÐµÑÐµÑÐ±ÑÑÐ³ÑÐºÐ¸Ð¹ Ð¿Ð¾Ð»Ð¸ÑÐµÑÐ½Ð¸ÑÐµÑÐºÐ¸Ð¹ ÑÐ½Ð¸Ð²ÐµÑÑÐ¸ÑÐµÑ ÐÐµÑÑÐ° ÐÐµÐ»Ð¸ÐºÐ¾Ð³Ð¾
Year: 2022
Subjects: ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ° Ð¸Ð³Ñ
Ð¼Ð°ÑÐ¸Ð½Ð½Ð¾Ðµ Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ
Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ Ñ Ð¿Ð¾Ð´ÐºÑÐµÐ¿Ð»ÐµÐ½Ð¸ÐµÐ¼
Ð²Ð½ÑÑÑÐµÐ½Ð½Ð¸Ðµ Ð½Ð°Ð³ÑÐ°Ð´Ñ
Ð»ÑÐ±Ð¾Ð¿ÑÑÑÑÐ²Ð¾
game development
machine learning
reinforcements learning
intrinsic rewards
curiosity

description: ÐÑÐ½Ð¾Ð²Ð½Ð°Ñ Ð¸Ð´ÐµÑ Ð¿Ð¾Ð´ÑÐ¾Ð´Ð° Ð·Ð°ÐºÐ»ÑÑÐ°ÐµÑÑÑ Ð² ÑÐ¾Ð¼, ÑÑÐ¾ ÐµÑÐ»Ð¸ ÐµÑÑÑ Ð²Ð½ÐµÑÐ½Ð¸Ðµ Ð½Ð°Ð³ÑÐ°Ð´Ñ, ÑÐ¾ Ð´Ð¾Ð»Ð¶Ð½Ñ Ð±ÑÑÑ Ð¸ Ð²Ð½ÑÑÑÐµÐ½Ð½Ð¸Ðµ. ÐÐ¼ÐµÑÑÐ¾ ÑÐ¾Ð³Ð¾ ÑÑÐ¾Ð±Ñ Ð¿ÑÐµÐ´Ð¾ÑÑÐ°Ð²Ð»ÑÑÑÑÑ ÑÑÐµÐ´Ð¾Ð¹, Ð²Ð½ÑÑÑÐµÐ½Ð½Ð¸Ðµ Ð²Ð¾Ð·Ð½Ð°Ð³ÑÐ°Ð¶Ð´ÐµÐ½Ð¸Ñ Ð³ÐµÐ½ÐµÑÐ¸ÑÑÑÑÑÑ ÑÐ°Ð¼Ð¸Ð¼ Ð°Ð³ÐµÐ½ÑÐ¾Ð¼ Ð½Ð° Ð¾ÑÐ½Ð¾Ð²Ðµ Ð½ÐµÐºÐ¾ÑÐ¾ÑÑÑ ÐºÑÐ¸ÑÐµÑÐ¸ÐµÐ², ÐºÐ¾ÑÐ¾ÑÑÐµ Ð² Ð¸ÑÐ¾Ð³Ðµ ÑÐ»ÑÐ¶Ð°Ñ ÐºÐ°ÐºÐ¾Ð¹-ÑÐ¾ ÑÐµÐ»Ð¸, Ð½Ð°Ð¿ÑÐ¸Ð¼ÐµÑ Ð¸Ð·Ð¼ÐµÐ½ÐµÐ½Ð¸Ðµ Ð¿Ð¾Ð²ÐµÐ´ÐµÐ½Ð¸Ñ Ð°Ð³ÐµÐ½ÑÐ° ÑÐ°ÐºÐ¸Ð¼ Ð¾Ð±ÑÐ°Ð·Ð¾Ð¼, ÑÑÐ¾ Ð¾Ð½ Ð¿Ð¾Ð»ÑÑÐ¸Ñ ÐµÑÐµ Ð±Ð¾Ð»ÑÑÐ¸Ðµ Ð²Ð½ÐµÑÐ½Ð¸Ðµ Ð½Ð°Ð³ÑÐ°Ð´Ñ Ð² Ð±ÑÐ´ÑÑÐµÐ¼, Ð¸Ð»Ð¸ ÑÑÐ¾ Ð°Ð³ÐµÐ½Ñ Ð±ÑÐ´ÐµÑ Ð¸ÑÑÐ»ÐµÐ´Ð¾Ð²Ð°ÑÑ Ð¼Ð¸Ñ Ð±Ð¾Ð»ÑÑÐµ, ÑÐµÐ¼ Ð¼Ð¾Ð³ Ð±Ñ Ð² Ð¿ÑÐ¾ÑÐ¸Ð²Ð½Ð¾Ð¼ ÑÐ»ÑÑÐ°Ðµ. ÐÑÐ»Ð¸ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐ°Ð½Ñ Ð¿ÑÐ¾ÑÐ¾ÑÐ¸Ð¿Ñ Ð¿ÑÐ¾ÑÐ¸Ð²Ð½Ð¸ÐºÐ¾Ð², ÐºÐ¾ÑÐ¾ÑÑÑ Ð¾Ð±ÑÑÐ¸Ð»Ð¸ Ð² ÑÐµÑÑÐ¾Ð²Ð¾Ð¹ ÑÑÐµÐ´Ðµ Ð¿ÑÐµÑÐ»ÐµÐ´Ð¾Ð²Ð°ÑÑ Ð¸ Ð°ÑÐ°ÐºÐ¾Ð²Ð°ÑÑ Ð³Ð»Ð°Ð²Ð½Ð¾Ð³Ð¾ Ð³ÐµÑÐ¾Ñ Ñ Ð¿Ð¾Ð¼Ð¾ÑÑÑ Ð¸Ð½ÑÑÑÑÐ¼ÐµÐ½ÑÐ°ÑÐ¸Ñ ÑÑÐµÐ´Ñ ÑÐ°Ð·ÑÐ°Ð±Ð¾ÑÐºÐ¸ Unity Ð´Ð»Ñ ÑÐ°Ð±Ð¾ÑÑ Ñ Ð¼Ð°ÑÐ¸Ð½Ð½ÑÐ¼ Ð¾Ð±ÑÑÐµÐ½Ð¸ÐµÐ¼ - ML-Agents. ÐÐ°ÑÐµÐ¼ Ð¿Ð¾Ð»ÑÑÐµÐ½Ð½ÑÐµ Ð¼Ð¾Ð´ÐµÐ»Ð¸ Ð°Ð³ÐµÐ½ÑÐ¾Ð² Ð¸Ð½ÑÐµÐ³ÑÐ¸ÑÐ¾Ð²Ð°Ð»Ð¸ Ð² Ð¾ÑÐ½Ð¾Ð²Ð½ÑÑ ÑÑÐµÐ½Ñ Ð¿ÑÐ¾ÐµÐºÑÐ° Ð² Ð²Ð¸Ð´Ðµ Ð½ÐµÑÐºÐ¾Ð»ÑÐºÐ¸Ñ ÑÐºÐ·ÐµÐ¼Ð¿Ð»ÑÑÐ¾Ð² Ð¿ÑÐ¾ÑÐ¸Ð²Ð½Ð¸ÐºÐ¾Ð², ÑÐ°ÑÐ¿Ð¾Ð»Ð¾Ð¶ÐµÐ½Ð½ÑÑ Ð² ÑÐ°Ð·Ð»Ð¸ÑÐ½ÑÑ Ð¼ÐµÑÑÐ°Ñ, ÐºÐ¾ÑÐ¾ÑÑÐµ Ð² Ð¿Ð¾ÑÐ»ÐµÐ´ÑÑÐ²Ð¸Ð¸ ÑÐµÑÐºÐ¾ Ð²ÑÐ¿Ð¾Ð»Ð½ÑÐ»Ð¸ Ð·Ð°Ð´ÑÐ¼Ð°Ð½Ð½ÑÐ¹ ÑÑÐ½ÐºÑÐ¸Ð¾Ð½Ð°Ð». Ð¢Ð°ÐºÐ¶Ðµ Ð±ÑÐ»Ð¾ Ð¿ÑÐ¾Ð¸Ð·Ð²ÐµÐ´ÐµÐ½Ð¾ ÑÑÐ°Ð²Ð½ÐµÐ½Ð¸Ðµ Ð¿ÑÐ¾ÑÐµÑÑÐ° Ð¸ ÑÐµÐ·ÑÐ»ÑÑÐ°ÑÐ¾Ð² Ð¾Ð±ÑÑÐµÐ½Ð¸Ñ Ñ Ð¿ÑÐ¸Ð¼ÐµÐ½ÐµÐ½Ð¸ÐµÐ¼ Ð¿ÑÐµÐ´Ð»Ð°Ð³Ð°ÐµÐ¼Ð¾Ð³Ð¾ Ð¼ÐµÑÐ¾Ð´Ð° Ð¸ Ð±ÐµÐ· Ð½ÐµÐ³Ð¾ - Ð¾Ð±ÑÑÐµÐ½Ð¸Ðµ, Ð¾ÑÐ½Ð¾Ð²Ð°Ð½Ð½Ð¾Ðµ Ð½Ð° Ð»ÑÐ±Ð¾Ð¿ÑÑÑÑÐ²Ðµ, Ð¿Ð¾ÐºÐ°Ð·Ð°Ð»Ð¾ Ð¿ÑÐµÐ²Ð¾ÑÑÐ¾Ð´ÑÑÐ²Ð¾.
description: The main idea of the approach is that if there are external rewards, then there should be intrinsic ones. Instead of being provided by the environment, intrinsic rewards are generated by the agent itself based on certain criteria that ultimately serve some purpose, for example, changing the agent's behavior so that he will receive even greater external rewards in the future, or that the agent will explore the world more than he could otherwise. Prototypes of opponents were developed, who were trained in a test environment to chase and attack the main character using the Unity development environment toolkit for working with machine learning - ML-Agents. Then the resulting agent models were integrated into the main stage of the project in the form of several instances of opponents located in various places, which subsequently clearly fulfilled the intended functionality. There was also a comparison of the process and learning outcomes with and without the proposed method - learning based on curiosity showed superiority.