## 5 Control Performance Evaluation of the DQN Model  
Comparative control algorithms were used to verify the effectiveness of the DQN model (controller). Skyhook and groundhook control algorithms were selected as comparative control algorithms, because they have typically been used for semi-active control problems and have shown good performance. A skyhook controller provides effective performance for dynamic-response reduction of the mass isolated from the vibrating base, such as in a semi-active base isolation system. A groundhook controller effectively decreases dynamic responses of the structure with the auxiliary mass, such as with a semi-active tuned mass damper [36]. Figure 12 shows the ideal configurations of the skyhook and  
Figure 11: History of MSE loss value. Figure 10: Calculation of MSE loss. groundhook controllers. These conceptual arrangement of an ideal skyhook and groundhook cannot be constructed in practice, because the damper cannot be connected to the sky or an absolute fixed base. The purpose of skyhook or groundhook controllers is to imitate the ideal structural configuration in Figure 12 using the practical SMIS shown in Figure 5b. [PERSON] et al. showed that the displacement-based on-off controller can provide the best performance for the benchmark problem among various versions of the skyhook and groundhook control algorithms [37]. The relative velocity between the roof story and the 11 th story, i.e., \(v_{1}-v_{2}\) in Figure 12, is an important value for the development of the skyhook and groundhook controllers. The displacement-based skyhook algorithm is formulated by the relative velocity and the absolute displacement of the upper isolated structure (x\({}_{2}\)). The on-off skyhook control algorithm selects the minimum or the maximum command voltage. This calculation is performed by using Equation (16):  
\[V=\left\{ \begin{aligned} & V_{\max}\text{ if }x_{2}(v_{1}-v_{2})\leq 0\\ & V_{\min}\text{ if }x_{2}(v_{1}-v_{2})>0\end{aligned} \right. \tag{16}\]  
where \(V\) is the control command voltage; \(V_{\max}\) is the maximum voltage, 5 V; and \(V_{\min}\) is the minimum voltage, 0 V. The displacement-based on-off groundhook control algorithm can be obtained by changing x\({}_{2}\) to x\({}_{1}\) in the skyhook controller. The groundhook controller can be expressed by:  
\[V=\left\{ \begin{aligned} & V_{\max}\quad\text{if }x_{1}(v_{1}-v_{2})\geq 0\\ & V_{\min}\quad\text{if }x_{1}(v_{1}-v_{2})<0\end{aligned} \right. \tag{17}\]  
After 10,000 episodes of reinforcement learning training, the DQN model was obtained to control the SMIS. Python 3.5.0 was employed to program the DQN model and environment generation codes, with Tensorflow 1.6.0 as a machine learning library. Time history analyses of the sample building structure with the SMIS were performed to investigate the seismic response control performance of the DQN model. An artificial ground acceleration not used for training was used for dynamic time history analysis. Figures 13-16 compare the sample structure's seismic response time histories using the SMIS controlled by the DQN model versus the skyhook and groundhook control algorithms. Figure 13 shows the inter-story drift time histories of three control cases at the 11 th story, where the peak story drift occurs. In this figure, the 11 th-story drift controlled by the groundhook controller is generally greater than those of the other controllers. The DQN model developed in this study provided much better control performance, in comparison with the two typical semi-active controllers. In particular, the 11 th-story drift between  
Figure 12: Idealized semi-active control configurations: (**a**) skyhook, (**b**) groundhook. about 7 and 10 s, when the response amplitude is significant for both the EV1 and EV2 earthquakes, was effectively reduced by the DQN model. Figure 14 presents isolation story drift time histories of the three controllers.