Abstract. Testing is the most dominant validation activity used by industry today, and there is an urgent need for improving its effectiveness, both with respect to the time and resources for test generation and execution, and obtained test coverage. We present a new technique for automatic generation of real-time black-box conformance tests for non-deterministic systems from a determinizable class of timed automata specifications with a dense time interpretation. In contrast to other attempts, our tests are generated using a coarse equivalence class partitioning of the specification. To analyze the specification, to synthesize the timed tests, and to guarantee coverage with respect to a coverage criterion, we use the efficient symbolic techniques recently developed for model checking of real-time systems. Application of our prototype tool to a realistic specification shows promising results in terms of both the test suite size, and the time and space used for test generation. Communicating and embedded real-time systems remain among the most challenging class of systems to develop correctly. The challenge arises in part from the inherent complexity in these systems, but in particular from the lack of adequate methods and tools to deal with this complexity. This applies to most development activities, including specification, design, and implementation. Testing consists of executing a program or a physical system with the goal of finding errors. It is not unusual to spend more than a third of the total development time on testing in industrial projects, and it therefore constitutes a significant portion of the cost and the time to market of the product. Because testing is the most dominant validation activity used by industry today, there is an urgent need for improving its effectiveness, both with respect to the time and resources used for test generation and execution, and also the obtained coverage. A potential improvement that is being examined by researchers is to make testing a formal method, and to provide tools that automate test case generation and execution. This approach has experienced some level of success: formal specification and automatic test generation are being applied in practice [10, 30, 33, 37] , and academic as well as commercial test generations tools are emerging for specification languages like SDL (Specification and Design Language) [11, 25, 29, 38] , CSP (Communicating Sequential Processes) [34], Lotos, and Promela [21]. However, current tools do not address real-time systems, or only provide a limited support of testing the timing aspects. They often abstract away the actual time at which events are supplied or expected, or do not select these time instances thoroughly and systematically. A large varitey of testing types and techniques exist, and they may be classified by the visibility of the implementation (black box or white box), the granularity of the implementation (function, component, system level), and by the correctness aspect being examined (robustness, reliability, performance, or behavior). This paper contributes new techniques that will enable automatic generation of black box test cases that check the external timing behavior of real-time systems or components.
The test generation method proposed in REF consists of two steps.
Automated test generation from timed automata
International Journal on Software Tools for Technology Transfer
International Journal on Software Tools for Technology Transfer
Computer Science