## 2 Materials and Methods
### Reinforcement Learning in Continuous Environments
The number of nodes in the third hidden layer is 32 (reduced size to allow gradual compression of features), and the activation function is also ReLU. The output layer has 7 nodes. Each node in the output layer represents an action or Q-value. If this is a Q-network for a discrete action space, each output corresponds to the Q-value of an action, indicating the expected cumulative reward of each action given the current state as can be seen in Table 2.

This architecture provides a balance between capacity (through multiple hidden layers) and efficiency, making it suitable for the RL tasks that require capturing complex patterns in state-action spaces. RL revolves around four key components: the policy, reward function, value function, and environmental model. The fundamental concept is that an intelligent agent must learn to perform a series of actions to maximize cumulative rewards through \begin{table}
\begin{tabular}{c c c} \hline \hline
**Input**: \(\text{Y}_{\text{demo}}\) //demonstrated path \\ Initiate \(n_{dmp}=2,n_{bfs}=10,d_{t}=0.02,\tau=1,\alpha z,\beta z,\alpha x,c,\sigma\) \\
**for each** time interval \(t=1:\tau/d_{t}\) \\ \(x=x(t-1)-\alpha x\cdot x_{t}\cdot d_{t}\) \\ \(\psi_{1}(\text{x})=\exp\bigl{(}-\frac{1}{2\sigma_{t}^{2}}(\text{x}-\text{c}_{1 })^{2}\bigr{)}\) \\
**end** \\ \(y_{0}=y_{0_{\text{demo}}}\) \(\delta=g_{\text{demo}}\) \\
**for each**\(n\_{dmp}\)//learning weight \\ \(\text{f}_{\text{target}}=\bar{\text{y}}_{\text{demo}}-\alpha z(\beta z(\text{g}- \text{y}_{\text{demo}})-\dot{y}_{\text{demo}})\) \\ \(\text{W}_{1}=\frac{\bar{\text{s}}\Gamma_{1}\text{f}_{\text{target}}}{\bar{ \text{s}}\Gamma_{1}\text{S}},S=\begin{pmatrix}x(t_{0})(g-y_{0})\\ x(t_{1})(g-y_{0})\\ \vdots\\ x(t_{N})(g-y_{0})\end{pmatrix},r_{1}=\begin{pmatrix}\Psi_{1}(t_{0})&0&\cdots &0\\ 0&\psi_{1}(t_{1})&\cdots&0\\ \vdots&\vdots&\cdots&\vdots\\ 0&0&\cdots&\psi_{1}(t_{N})\end{pmatrix}\)/weight_init \\ \hline
**end** \\
**for** iteration \(w_{i}\) \\
**for** time interval \(x_{i}\in x\) \\
**for each**\(i=1:n_{dmp}\) \\ \(f(x)=\frac{\sum_{i}^{N}\frac{\psi_{i}(x)w_{i}}{\sum_{i}^{N}\dot{\psi}_{i}(x)}x(g -y_{0})}{\sum_{i}^{N}\dot{\psi}_{i}(x)}x(g-y_{0})\) \\ \(\ddot{y}_{2}=\alpha z(\beta z(g-y_{i})-z+z\) \\ \(\dot{y}_{2}=\dot{y}+\ddot{y}\cdot d_{t}\) \\ \(y_{i}=y+\dot{y}\cdot d_{t}\) \\
**end** \\ \(D_{min}=min(\mathbf{D_{x}}-\mathbf{y})\)//“target function” \\
**end** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: NN Layer Structure. its interactions with the environment. The reward function is designed to combine various reward variables into a single output value, which serves as feedback for the agent to understand the incentivized behaviors. In this case, the robot is expected to follow the goal for as long as possible while avoiding dynamic obstacles. Consequently, the reward function is defined as a weighted sum of two components: the distance between the end-effector and the goal as well as the minimum distance between the robot manipulator and nearby obstacles. Additionally, a negative reward is applied more heavily than a positive one to encourage the robot to avoid penalties, facilitating faster learning. ## 3 Results

The preliminary test result shown in the image depicts a MATLAB (R2024a) written program that describes a circle of radius \(r\). It can be observed that as the given trajectory is traversed, the joints assume their respective positions as depicted in Figure 5. The path points were recorded in variables in matrix form. They were then attached to the input port of the block. The path planner iterates the path points based on the given time parameters and generates an appropriate path. The result was obtained in the form of a position vector. These types of path planners not only calculate the path but also the velocity and acceleration. Polynomial trajectories are useful in cases where zero or non-zero velocity and acceleration segments need to be connected, as the acceleration profiles are much smoother.