Without quantitative metrics, there can be no basis for objective comparison and therefore no understanding. It is difficult to develop well-defined reliability, security or usability metrics, but the lack of these metrics may explain why so many computer systems are very fast, but unreliable, insecure, and unusable. If I were to go to the trouble of building a system with better usability than the alternative, I would want better evidence of that benefit than my intuition. Finally, I want to validate my results by using multiple methodologies with cross-validation. The investigative methodologies that I know of are theory, in situ measurement, and simulation.
Theory explains behavior through modeling and mathematical proofs.
In situ measurement explains behavior through statistical inferences from measurements taken from operational systems. Simulation explains behavior through analysis of a system that behaves like the investigated system. This figure shows different research methods as nodes in a graph. The directed edges of the graph indicate that one method is used to predict or indicate another method's results. For example, theory measurement means that theory is used predict measurement results.