Another aspect of this concerns research that looks at a single factor versus all the significant factors in a given situation. Applied projects must do the latter, and in so doing they point up the paucity of theoretical work in education that addresses the relationships between factors and so between theories. Applied projects force attention to this. Diagnosis vs. hypothesis testing While hypothesis testing is the main approach to theory-driven research, diagnosis is the main approach in applied work. A prior hypothesis in diagnosis is a bias often leading to bad conclusions, as in the systematic under-diagnosis of osteoporosis in men and heart problems in women because statistically they are more frequent in the other sex.
Diagnosis requires a focus on and respect for data, as opposed to one for theory.
This is all the more so as there is no reason to think we have all the theory we need. Thus diagnosis is not the easy task it is in closed or toy worlds, where you enumerate the possible causes and calculate an optimal tree of tests to identify which one it is this time. Whether in medicine, HCI, or education, diagnosis must be open to the real possibility that the problem is one not seen or understood before. That is how progress is made in applied fields. The only general approach, though it cannot be translated into a fixed procedure, is to lay oneself open as much as possible to surprises, and to clues independent of one's prior expectations.