Foundations of active automata learning: an algorithmic perspective

The wealth of model-based techniques in software engineering—such as model checking or model-based testing—is starkly contrasted with a frequent lack of formal models in practical settings. Sophisticated static analysis techniques for obtaining models from a sourceor bytecode representation have matured to close this gap to a large extent, yet they might fall short on more complex systems: be it that no sufficiently robust decision procedures are available, or that the system performs calls to external, closed source libraries or even remote web services. Active automata learning has been proposed as a means of overcoming this problem: by executing test cases on a system, finite-state machine models reflecting a portion of the actual runtime behavior of the targeted system can be inferred. This positions active automata learning as an enabler technology, extending the range of application for a whole array of formal, model-based techniques. Its usefulness has been proven in many different subfields of formal methods, such as black-box model checking, test-case generation, interface synthesis, or compositional verification. In a much-noted case study, active automata learning played a key role in analyzing the internal structure of a botnet with the aim of devising countermeasures. One of the major obstacles of applying active automata learning in practice is, however, the fact that it is a rather costly technique: to gain sufficient information for inferring a model, a large number of test cases need to be executed, which is also referred to as “posing queries. ” These test cases may be rather heavy-weighted, comprising high-latency operations such as interactions with hardware or remote network services, and learning systems of moderate size may take hours or days even when using algorithms with polynomial query complexities. The costliness of the technique calls for highly efficient algorithms that do not waste any information. The reality is surprisingly different from that ideal: many active automata learning algorithms that are being used in practice—including the well-known L∗ algorithm, which was the first one with a polynomial query complexity—frequently resort to heuristics to ensure certain properties, resulting in an increased overall query complexity. However, it has rarely been investigated why or even if these properties are necessary to ensure correctness, or what violating them entails. Related is the observation that descriptions of active automata learning algorithms are often less-than-formal, and merely focus on somehow arriving at a correctness proof instead of motivating and justifying the single steps. It is one of the stated goals of this thesis to change this situation, by giving a rigorously formal description of an approach to active automata learning that is independent of specific data structures or algorithmic realizations. This formal description allows the identification of a number of properties, some of which are necessary, while others are merely desirable. The connection between these properties, as well as possible reasons for their violation, are investigated. This leads to the observation that, while for each property there is an existing algorithm maintaining it, no algorithm manages to simultaneously maintain all desirable properties. Based on these observations, and exploiting further insights attained through the formalization, a novel active automata learning algorithm, called TTT, is developed. The distinguishing