# A Methodology for the Design of Application-Specific Cyber-Physical Social Sensing Co-Simulators
## 4 Experimental Validation: Co-Simulator Development and Experiment Description
### Experiment Description
#### 4.2.1 Non-Methodological Co-Simulator Implementation  
In all the described experiments, the results are compared to the values obtained from other additional co-simulators which have been implemented without following the proposed methodology. In particular, in order to carry out the experimental validation three additional co-simulators were implemented. These new co-simulators were developed following existing proposals about this topic, instead of following the proposed methodology. The first non-methodological co-simulator (hereinafter called \"simulator#1\") it was implemented following a kind of choreographed co-simulation paradigm, where every domain-specific simulator is recovered by a middleware being able to communicate with the rest of simulators [132]. Simulation models were maintained as defined by default in the domain-specific simulators, and no additional interface was deployed. All the domain-specific simulators are running in parallel. Graphics and results were shown using the instruments provided by NS3 simulator. Figure 14a describes the architecture of this new co-simulator. The simulation update process follows an iterative paradigm, where each simulation step implies a convergence phase. Every domain-specific simulator has to calculate in an independent way a first estimation of the situation of the scenario under study in  
Figure 13: Simulator’s user interfaces. (**a**) Basic configuration. (**b**) Simulation execution and control. (**c**) Simulation design. (**d**) Results presentation (temporal animation). the next step, which it is shared with the rest of the tools later. Considering this new information, domain-specific tools modify their initial calculations and the sharing and updating process is repeated until values converge to a stable situation. The second additional co-simulator (hereinafter called \"simulator#2\") was implemented following one of the most popular schemes for first concept proofs in research. It is similar to an \"orchestrated by a one of the simulators\" co-simulation paradigm, where the most versatile domain-specific tool controls and manage the entire simulation [115]. As in the base work, we have selected as main simulator the MATLAB suite. As in simulator#1, simulation models were maintained as defined by default in the domain-specific simulators, and no additional interface was deployed. A \"stop-and-wait\" coordination mechanism was applied and graphics and results were shown using the instruments provided by MATLAB. Figure 14b describes the architecture of this second new co-simulator. Finally, a third additional non-methodological co-simulator was implemented. This final co-simulator (hereinafter called \"simulator#3\"), it is identical to the methodological one. However, in this case, a parallel execution coordination mechanism is selected, instead of the stop-an-wait mechanism employed in the methodological simulator. With this structure, this co-simulator is perfect to evaluate, in a comparative way, the performing of the selected coordination mechanism (third step in the methodology). On the other hand, in order to allow comparisons between the proposed application-specific co-simulators and state-of-the-art tools, a benchmark simulator is also considered. As said in Section 2, no simulator proposed in the state-of-the-art allows simulating CPSS considering all subsystems in the same detail level (for example, if a fire is simulated, a realistic evolution depending the scenarios has to be followed, but also people's behavior has to be adapted to this situation and the impact in communications should be also evaluated--for example interferences in radio channels due to smoke-). In that way, benchmark domain specific simulators use to present a better performance (as said previously, simulations are faster and more scalable) because their simulations are simpler (coordination delays and congestion, for example, do not appear).