## 3 Research Infrastructure and Technology
The simulator in graphic mode, used to view the effects of navigation performed by the trained model, can show the values of individual sensors of the flying drone in real-time. As a library containing a set of ready-made implementations of RL algorithms (reinforcement learning), Stable baselines3 [40] written in PyTorch v2.3.1 technology was used. The goal of the creators of this module was the simplicity of using algorithms. The entire script in which the model of a given algorithm is initialized, trained, and tested can contain only a few lines of code. This allows the use of algorithms on an \"out of the box\" basis. The advantages of this solution include a typical structure for all algorithms, a standard programming style, and high-quality code. It allows for defining continuous sets of actions (Box) where the agent can perform actions described in ranges; in a given step, the action is not a single value but a vector created from defined component values. It also allows for defining discrete sets of actions (Discrete) where the number of actions the agent can take \begin{table}
\begin{tabular}{c c} \hline observation & An object that stores data about the environment defined by the user as observations. These can be pixels, velocities, or coordinates. \\ \hline reward & Information about the value of the reward for the previous action shows that the goal of the agent’s action is always to maximize the reward. \\ \hline done & A callback flag that tells you if the environment should be reset or not. The reason for restarting could be, for example, reaching a goal or leaving the board. \\ \hline info & Data is used during the testing of algorithms and in the learning process. They are not used by the agent when making decisions. \\ \hline \end{tabular}
\end{table} 
Table 2: OpenAI Gym action definitions for an agent (algorithm) performing successive actions. Figure 3: The idea of a test environment. is limited and strictly defined; during each move, the agent can perform only one move, selected from the list of available maneuvers. The Ryze Tello drone from DJI (Shenzhen DJI Sciences and Technologies Ltd., Nanshan, Shenzhen, China) was a critical tool for conducting experiments [41]. It was chosen due to its low price, high quality, and the possibility of programming using a ready API. This drone has numerous features, such as a built-in camera, automatically performing acrobatics, and image stabilization, making it ideal for research applications in limited spaces. It has 2.4 GHz Wi-Fi, a maximum speed of 28 km/h, a range of 100 m, a 5 Mpix camera, and a battery life of up to 13 min. API commands enable control such as takeoff, landing, rotation, acrobatics, and the ability to read the drone's status, such as speed, battery level, or altitude. The drone was used in closed spaces, which allowed for precise testing of algorithms in actual conditions while minimizing the risk of damage to the machine (Figure 4). The drone does not have a headless mode, i.e., movement in directions about the starting position. Setting the drone at the right angle before sending the command for the selected maneuver is essential. Figure 5 shows the location of the machine axes and the markings of the movements performed; these are for the Z Yaw axis, the Y Pitch axis, and the X Roll axis, respectively. It has a ready API that allows you to create your flight route setting scripts. The smartphone app has ready flight modes programmed by the creators. The drone is connected via a wireless network. The machine provides its access point, which must be connected to a computer acting as a controller.