What I'm calling for is for economists to drop the pretense and misconception that using empirical methods for studying macroeconomics makes it scientific. And I'm suggesting that being good at math is not useful unless you're using useful data. In the study of logic, arguments can be considered valid if they are formally correct, but still unsound if their premises are false. Much in the same way, one can perform any number of valid mathematical transformations on data but still be left with unsound conclusions if those data were gathered incorrectly.

I am not saying that empiricism is inherently flawed, or that we should stop collecting economic data. And I I do not intend to advocate any particular school of economic thought here. All I'm advocating is that students be taught how to think critically about what they are being taught. So much of a modern economics education consists of looking at the changes in figures over time that very little is spent focused on a more general kind of reasoning.

The kind of reasoning I'm calling for is not easy to define. This is one of the tremendous advantages numbers have over argument in most minds. This kind of reasoning takes into account the notion that most of the information we obtain is not perfect or complete, and that many of our determinations are really judgment calls on what is more likely to be true. If empiricism is reasoning with your eyes, this is reasoning with your nose. It is a trained skill that allows you to recognize dubious premises and unspoken assumptions. When refined, it allows you to distill the essence of arguments down to a set of axioms that you can use to build a coherent model of the situation at hand. It is this theoretical side that allows you to understand how to construct experiments that test hypotheses, or whether that is even possible in each case.

To demonstrate the importance of gaining an understanding of the theory and rules behind something before testing it, I offer a parable:

The commissioner of the NFL once decided that teams were punting too much and he hired an econometrician (economic statistician) to study the situation and provide a solution to this problem. The econometrician applied his skills to the task at hand, aggregating data from several seasons to find correlations. He noted that there is an incredibly strong correlation between forth downs and punting, and he recommended that the commissioner ban fourth downs. In the next season, offenses were only given three downs. To the econometrician's surprise and the commissioner's chagrin, teams actually punted more frequently, as the fewer number of downs dramatically limited offensive opportunities.

The econometrician's misunderstanding was based on something rather obvious (if you understand American Football): a failure to separate correlation and causation due to an ignorance of the rules of the game. And compared to a global economy, football is a very simple game, with very simple rules. Applying reasoning to the example is very straightforward, but applying the same thing to a world of dynamic human behavior is much more subtle. Which is why students ought to be trained to question assumptions and sense where logic and math have separated themselves from the reality they are supposed to help us describe.

People will disagree about when things correlate to reality, and about what things make sense in parables. But almost anyone can learn to recognize when a number seems too specific, just like most decent coders learn to recognize "code smell." Just the other day, someone told me confidently that 65% of communication is non-verbal. Now, while I almost agree intuitively, I immediately asked where they heard that, and how someone could have arrived at that figure, which seemed oddly specific for something (communication) that I don't think is frequently quantitized. Every student of a soft science needs to have this skill strongly developed, or they will begin to take these kinds of things at face value.