clean up and add more tests

this implements a number of new tests to better exercise the api, moves each
test into a separate function to better organize them, and increases the
DOT_GRAPH_MAX_NODES doxygen variable to accommodate the resulting larger call
graphs.