It is important to remember that a test is not reliable or unreliable. Thus, authors should provide reliability coefficients of the scores for the data being analyzed even when the focus of their research is not psychometric. Interpreting the size of observed effects requires an assessment of the reliability of the scores. Besides showing that an instrument is reliable, we need to show that it does not correlate strongly with other key constructs. It is just as important to establish that a measure does not measure what it should not measure as it is to show that it does measure what it should.
Researchers occasionally encounter a measurement problem that has no obvious solution.
This happens when they decide to explore a new and rapidly growing research area that is based on a previous researcher's well-defined construct implemented with a poorly developed psychometric instrument. Innovators, in the excitement of their discovery, sometimes give insufficient attention to the quality of their instruments. Once a defective measure enters the literature, subsequent researchers are reluctant to change it.