Look before you leap: on some fundamental issues in software engineering research
In science, clear definition and rigorous representation are of prime importance. Recently, Briand et al. (IEEE TSE 22 (1) (1996) 6885) proposed to clarify ambiguous software quality concepts such as size and complexity through a definition based on their intuitive properties. A seemingly rigorous and generic system representation, suitable for analyzing software described with existing tools, is also advanced. We justify that their property-based definitions for these concepts are logically untenable and practically misleading. We demonstrate further that their representation is still vague and searching for a general system representation is doomed to failure and cases of refutation are provided. In addition, we prove that all the system definitions, even for the very limited practical case of programs, are flawed. Finally, we elucidate with Briand et al.'s examples that theoretic representation must be validated, set theory seems incompatible with programs, and reasoning by analogy could be misleading. Our in-depth analysis illustrates that formality is but skin-deep and we must be aware of the danger of ignoring the most important methodological rigor in software engineering research.
definition representation validation research methodology