OpenSourcePolicyCenter/Tax-Calculator — pull request #65 by Copper-Head (2014-11-11 20:38:01+00:00)
Title: More verbose testing

I've finally had a chance to sit down and start looking at ways to make our testing scheme "smarter".
I'm not done with it by any means, but would like to hear feedback and opinions before I go down some rabbit hole and waste too much time with useless prototypes. This is low priority compared to refactoring.

The proposal in short is as follows:
We use [networkx](https://networkx.github.io/) to load a graph of all our variable dependencies from a JSON file. As soon as we detect a discrepancy between the values of a variable in the gold standard vs. the testing output, we look up the variable label in the graph and recursively search through its ancestors and descendants for more errors. In the end we get a printout of the culprit variable and whichever ancestors/descendants also contain discrepancies.
Pros:
* After an initial time investment of creating the graph by going through the code with a comb, tracking down a bug should be significantly easier.
* Using JSON's universality as input, we could visualize the data flow in the tax logic, which could be useful for educational purposes.
Cons:
* While I already have detailed records for some variable dependencies left over since the last bug squashing session, we will have to invest quite a bit of time in filling the remaining gaps.
* The JSON format, while readable to humans, is currently not very conducive to manual alterations, which means we'll have to use the networkx interface to update the graph in case the logic changes.