## 3 Establishment of CPPS Real-Time Hardware-in-the-Loop Co-Simulation Platform
### Sequential Procedure and Control Logic of the Whole Platform  
The key goals of the control loop were to realize the synchronization of different simulators, modules, or processes that participate in the co-simulation to ensure the timeliness and accuracy of message delivery, and achieve a combination of independent simulation and collaborative simulation according to demand. Therefore, the distributed simulation components could be coordinated in a unified environment with high interoperability and reusability.  
The sequential procedure and control logic are shown in Figure 4 where the master station of the control center is the bridge that implements the control logic and time synchronization between both simulators. After receiving, analyzing, and handling data produced in the power model and communication model, the center generates control commands accordingly, and transfers them back to the power and communication environments. Power and information systems can be analyzed and controlled synchronously to meet the requirements of co-simulation.  
Aiming at information acquisition and safety control service, the platform realizes the closed-loop data flow via two sets of communication networks where the blue line represents the data flow and the control flow between the power network and the master station system, and the red line represents the flow between the master station and the stability control device. In the wide area load control service researched in this paper, the most important feature in the system was its delay parameters. Therefore,  
Figure 4: Framework of CPPS real-time hardware-in-the-loop co-simulation platform.  
the delay parameters of each part and their interactive influences were analyzed as shown in Figure 5. For a complete closed-loop data flow, the total delay can be expressed as follows.  
\[\Delta t_{d}=\Delta t_{n}+\Delta t_{s}+\Delta t_{p}+\Delta t_{i} \tag{1}\]  
where \(\Delta t_{d}\) is the total delay of the platform, namely the time delay from the moment when the fault happens, to the moment when the control device acts. It is made up of several parts including the inherent delay \(\Delta t_{i}\), sampling delay \(\Delta t_{s}\), processing delay \(\Delta t_{p}\), and network transmission delay \(\Delta t_{n}\). The inherent delay \(\Delta t_{i}\) is related to the hardware parameters and the operating time inside each device, which is a constant value independent of data size. The sampling delay \(\Delta t_{s}\) is the time delay from the moment when packets arrive at the device to the moment when packets are received, and is subject to uniform distribution from 0 to the sampling period. The processing delay \(\Delta t_{p}\) is the analyzing and processing time of data, which is related to the packet format and the device's computing ability. Moreover, it shows a positive correlation with the sampling rate. With the increase in sampling rate, the queuing time will rise, and the processing delay will also increase accordingly. The network transmission delay \(\Delta t_{n}\) is related to the parameters and congestion degree of the communication lines, which is simulated in the OPNET communication network.  
The measurement of inherent delay is described as follows. The other delay parameters are set to an ideal situation, and the time factor of simulation is set to 1 (i.e., equal to real-world time). Through a simple data sending and receiving process without data processing steps, a 120-s long simulation test was carried out as shown in Figure 6. The average of inherent delay was 0.588 ms with little jitter, which was much smaller than the transmission delay of the communication service. Therefore, the simulation error resulting from inherent delay was limited without affecting the simulation accuracy.  
Figure 5: Time synchronization process.  
Figure 6: Inherent delay of the co-simulation platform.