# Reward Function and Configuration Parameters in Machine Learning of a Four-Legged Walking Robot
## 1 Introduction
### Literature
Such a definition of its operation ensures significant efficiency in controlling multidimensional processes. Despite the generally defined scheme of the artificial neural network, we can distinguish several of its types. The first, and most basic, of these is the one-way network. There is no feedback; that is, each piece of information passes through each neurone of the network only once. In addition, they can be differentiated into single-, double-, and multi-layer networks. Other types are recursive networks (e.g., Hopfield networks and Boltzmann machines). We can call such a network a creation in which the connections between neurones constitute a graph with cycles. Another type of ANN worth mentioning is self-organising maps, called Kohonen networks. In their case, the corresponding coordinates placed in any n-dimensional space are juxtaposed with the neurones. In addition to the networks mentioned above, there are other networks, such as support vector machines, networks based on radial basis functions, token-transmitting networks, and networks based on impulsive neurones [15]. Owing to their versatility and an ever-wider spectrum of possibilities, artificial neural networks have found many applications in an increasing number of fields including:  
* recognition, denoising, and classification of graphics (including letters);
* speech synthesis;
* in the aviation industry, to identify dangerous items in luggage;
* data analysis, especially finding correlations between data series;
* estimation, prediction, and approximation of the output based on inputs, without detailed knowledge of the process [16; 17]. The current state of technology, in the context of teaching methods using artificial neural networks, distinguishes three basic trends:  
* supervised learning;
* unsupervised learning;
* reinforcement learning. In the study that is the subject of this article, the third type was used. Reinforcement learning is a type of dynamic programming for which the task is to train algorithms through a system of punishments and rewards. This means that the agent undergoing the learning process is tasked with minimising the punishments received and maximising the rewards based on entering appropriate interactions with the environment. The main difference, compared to the other two mentioned methods, is the lack of a prepared set of training data, but only in the configured environment [18]. Reinforcement learning can be represented by a mathematical model of the Markov Decision Process (MDP). \[\text{MDP}=<X,A,\varrho,\delta> \tag{2}\]  
where \(X\) is a finite set of states, \(A\) is a finite set of actions, \(\varrho\) is a reward function, and \(\delta\) is a state transition function. Figure 1: Model of the described neurone. To fully understand the essence of reinforcement learning, it is necessary to describe the three most important components of the process. The first is the environment, that is, the simulation with which the agent interacts [19; 20]. It is characterised by the following features:  
* state--according to the name, it is a variable that defines the current state of the environment;
* step--a function which, based on the performed action, updates the state of the environment and returns a reward and an observation;
* episodes--a set of steps that, upon being reached, resets the state of the environment;
* reward--a variable returned by the environment, specifying the benefit (or loss) obtained after a given step;
* observation--a scalar, vector, or matrix returned by a step for which the task is to describe the state of the environment at a given moment. The second part of the reinforcement learning process is the agent, that is, the element that interacts with the environment. As described above, its task is to determine the most favourable way to interact with the environment. The basic parameter of the agent is the function responsible for its behaviour, accepting the observation, and returning the action [21]. Last but not least, the most important part of the training process is the buffer.