## 3 Dynamic Model Description and Numerical Experiments' Setup  
Recently, a plethora of benchmark problems has been used to evaluate the performance of RL algorithms. For instance, the Arcade Learning Environment [63, 64] for discrete action space problems or OpenAI Gym [65], which considers benchmark control problems. Among the latter, there is the cart-pole swing-up [66], an underactuated, multivariable, inherently unstable, and strongly coupled nonlinear problem, which has been studied in the past by several authors [67]. When it comes to RL, the moderate level of complexity of the cart-pole system results in being advantageous due to its potential generalization to different domains, in contrast with other RL benchmarks such as the locomotion task [68, 69], which requires highly specific algorithmic adaptations [70]. In this paper, the said system is chosen as a case study because its topological simplicity allows for performing a detailed study of the sensitivity of the developed controller in the presence of dry friction forces. Figure 2 shows the scheme of the system, consisting of two rigid bodies, the cart with mass \(m_{c}\), which moves along the \(X\) axis, and the pole with mass \(m_{p}\), its centroidal moment of inertia \(l_{zz,p}\) on the \(Z\) axis, and the center of mass positioned at a distance \(L\) from the revolute joint that allows the  
pendulum to rotate freely. The Lagrangian coordinates of the system are set as \(\textbf{q}=\left[ \begin{array}{cc}x&\theta\end{array} \right]^{T}\), and the corresponding equations of motion can be written as follows:  
\[\textbf{M}\ddot{\textbf{q}}=\textbf{Q}_{b} \tag{5}\]  
where \(\ddot{\textbf{q}}\) is the second time derivative of **q**, whereas **M** and \(\textbf{Q}_{b}\) are respectively equal to:  
\[\textbf{M}=\left[ \begin{array}{cc}m_{c}+m_{p}&-Lm_{p}\cos\left(\theta\right) \\ -Lm_{p}\cos\left(\theta\right)&m_{p}L^{2}+l_{zz,p}\end{array} \right] \tag{6}\]  
\[\textbf{Q}_{b}=\left[ \begin{array}{cc}-\dot{\theta}^{2}m_{p}L\sin\left( \theta\right)\;+F_{f}+F_{c}\\ m_{p}gL\sin\left(\theta\right)\end{array} \right] \tag{7}\]  
where \(F_{c}\) is the horizontal force the controller applies on the cart with the goal to swing-up the pole and \(F_{f}\) is the friction force. The modeling of dry friction in mechanical systems' simulation plays a fundamental role. Considering this phenomenon in the early stages of engineering design is crucial to reduce the gap between numerical simulation and experimental results [71]. Multiple approaches to model dry friction exist in the literature, yet none of them has general validity [72]. The classical Coulomb dry friction model is widespread in engineering applications. The model considers that friction force opposes the motion and that its magnitude is independent of the contact area and velocity. Therefore, the dry friction force can be computed as:  
\[F_{coul}=-\mu\left|F_{N}\right|sgn(\dot{x}) \tag{8}\]  
where \(\mu\) is the dry friction coefficient, \(F_{N}\) is the instant normal force produced by the contact between the two surfaces interacting, and \(\dot{x}\) is the relative velocity between them, whereas \(sgn\) is the sign function defined as:  
\[sgn(x)=\left\{ \begin{array}{cc}-1,&x<0\\ 0,&x=0\\ 1,&x>0\end{array} \right. \tag{9}\]  
Figure 2: The cart-pole system. Despite the apparent straightforwardness of the Coulomb friction model, for the implementation into dynamical systems simulation, it results in being challenging for numerical integration [71]. This is due to the friction force discontinuity at zero velocity, producing stiff equations of motion and the nonuniqueness and nonexistence of the solution for accelerations [72]. As a solution, the so-called smooth Coulomb friction model has been proposed [73]. It considers a smoothing function to remove the discontinuity. Multiple smoothing functions, such as linear, exponential, or trigonometric functions, have been employed in the literature. However, the smooth Coulomb friction model does not fully capture the physical phenomenon.