# Vibration Control with Reinforcement Learning Based on Multi-Reward Lightweight Networks
## 3 Method
### Multi-Reward Mechanism  
The multi-reward mechanism is a technology in reinforcement learning that is used to introduce and process multiple reward signals more effectively to enhance the learning and decision-making ability of agents. In some complex environments, a single reward signal may not fully express the diversity and complexity of the environment. Therefore, the introduction of multiple reward signals helps to provide more comprehensive and accurate training signals to improve the learning process and strategies. Compared with the previous method [26], the reward function is as follows:  
\[r_{OM}=-\Big{(}x_{1}^{2}+x_{2}^{2}+x_{3}^{2}+x_{4}^{2}\Big{)} \tag{33}\]  
We propose three other reward functions, as follows (34)-(36):  
\[r_{MSV}=-\frac{1}{N}.\sum_{i=1}^{N}\Big{(}x_{1i}^{2}+x_{2i}^{2}+x_{3i}^{2}+x_ {4i}^{2}\Big{)} \tag{34}\]  
As shown in Formula (34), considering the mean square of four sensors over a period of time as the reward signal helps to reduce noise and fluctuations, making it smoother. This aids the agent to better understand the long-term trends of the environment, avoiding excessive interference from individual time-step rewards, and improving the stability and reliability of training. This approach also helps the agent to better consider long-term gains, reinforcing awareness of long-term planning and decision making rather than just focusing on immediate rewards. Averaging the reward signal also reduces the impact of environmental noise and randomness, enhancing the agent's ability to handle complex situations. Overall, adopting the average value over a period of time as the reward function can improve training stability, smooth the reward signal, better consider long-term gains, and enhance the agent's resistance to interference, effectively improving the training effectiveness of reinforcement learning models. Figure 8: The DRSN_CW with 18 layers. Our Actor network is influenced by the ResNet architecture, with differences in each layer’s convolutional kernels, stride, and channels, and an added shrinkage  
component. Specifically, the input data dimensions are [B, 1, 4, 100], where B represents the batch size, 4 indicates the four sensors, 100 signifies 100 time dimensions, and 1 denotes a reserved specific feature dimension. The data pass through a red-colored convolutional layer with a kernel size of [1, 3], a stride of 1, an input channel of 1, and an output channel of 4, resulting in data dimensions of [B, 4, 4, 100]. The setting for convolutional layers within the same-colored modules remains consistent. Every two convolutional modules combined with the shrinkage component form an RSBU-CW construction unit, where units of the same color share the same network settings. \[ \begin{array}{c}\tau_{RMSE}=-\frac{1}{M}\sum\limits_{j=1}^{M}\sqrt{\frac{ \sum_{i=1}^{N}x_{j}^{2}}{N}}\\ \\ =-\frac{1}{M}\left(\sqrt{\frac{\sum_{i=1}^{N}x_{i1}^{2}}{N}}+\sqrt{\frac{\sum_{ i=1}^{N}x_{i2}^{2}}{N}}+\sqrt{\frac{\sum_{i=1}^{N}x_{i2}^{2}}{N}}+\sqrt{\frac{ \sum_{i=1}^{N}x_{i1}^{2}}{N}}\right)\end{array}  \tag{35}\]  
As shown in Formula (35), adopting the average of the root mean square (RMSE) of four sensors as the reward function has several advantages. Firstly, compared to extreme values, it exhibits higher robustness and stability. This means that the RMSE has less impact on the reward function when dealing with outliers, more evenly considers the overall characteristics of the data, and is less affected by extreme values. Additionally, the RMSE, being the square root of the mean square error, provides a more intuitive and understandable measurement. Compared to the mean square error, RMSE offers a more intuitive description of the error magnitude, emphasizing the size of the average error rather than the square of errors. It is noteworthy that the RMSE is less influenced by noise because it takes the square root of the mean error, resulting in a smoother signal.