A Checklist for Interpreting Statistics

Often, in response to some headline figure, summary stat, number, figure, etc, there is some qualification, adjustment or different interpretation for the headline that substantially changes the meaning.
For example, see this piece by Matthew Yglesias that provides a different look at the same data that produces a totally different interpretation. 
Is it possible to write a sort of checklist of the causes of different interpretations? E.g.
* Sample selection bias
* Compositional changes by var x, where var x is correlated to the magnitude of the conclusion.
* etc?