# MBSE Testbed for Rapid, Cost-Effective Prototyping and Evaluation of System Modeling Approaches
## 5 Discussion  
This paper has presented the system concept, architecture, prototype implementation, and quantitative analysis supported by a MBSE testbed that enables experimentation with different models, algorithms, and operational scenarios. Several important lessons were learned from the prototype implementation. First, a minimal testbed ontology [24] with essential capabilities can be quite useful to begin initial experimentation with models and algorithms. Second, it is possible to adapt system model complexity to scenario complexity and thereby minimize computation load when possible. For example, for simple driving scenarios, a finite state machine (FSM) can suffice for vehicle navigation. However, as the driving scenario gets more complicated (e.g., poor observability and uncertainties in the environment), more complex system models such as POMDP can be employed to cope with environment uncertainty and partial observability. The value of POMDP modeling becomes evident when operating in complex uncertain environments. Third, when it comes to system development, it pays to start off with simple scenarios and ensure that safety requirements are met, and then progressively complicate driving scenarios and employ more complex models while continuing to assure satisfaction of safety requirements. The most straightforward way to implement this strategy is to control the complexity of the operational environment by imposing constraints (e.g., have a dedicated lane for autonomous vehicles or avoid driving in crowded streets). Then, after the simple model has been shown to satisfy safety requirements, constraints can be systematically relaxed to create more complex driving scenarios. In the latter case, more complex system models can be employed, verified, and validated with respect to safety requirements. Fourth, system and environment models can be reused, thereby reducing development time. To facilitate model reuse, the models can be metadata-tagged with usage context. Then, contextual similarity between a problem situation and system models can be employed to determine  
Figure 21: Model-specific parameter evaluation. suitability of a particular system model for reuse in a particular context. This reuse feature can accelerate experimentation and development. Fifth, a smart, context-sensitive, scenario-driven dashboard can be used to dynamically adapt monitoring capability and dashboard display to maximize situation awareness with manageable cognitive load. To this end, the familiar METT-TC construct employed by the military can be employed as the underlying ontology. Based on the prevailing context, either the total set or a subset of variables in METT-TC may be applicable to characterize context. The flexible architecture of the scenario-driven dashboard can support both human-in-the-loop and autonomous operations. Just as importantly, it provides a convenient and cost-effective environment to try out different augmented-intelligence concepts [25]. Sixth, a hybrid, distributed simulation capability enables integration of virtual simulation with real-world systems (e.g., self-driving cars and unmanned aerial vehicles). This integration, in turn, enables the creation of digital twins which can enhance system verification and validation activities [5]. Furthermore, by assuring individual control of simulations, the testbed offers requisite flexibility in experimentation with system/system-of-system simulation. In addition, by allowing different models (e.g., scenario model and threat model) to run on different computers, simulation performance can be significantly increased. Finally, an illustrative quantitative analysis capability is presented to convey how simulation results can be analyzed to generate new insights.