defaultdict-ectomy, for Python 2.4

Because Python 2.4 doesn't have collections.defaultdict,
and because we are barely using the power of defaultdict
(we can replace both instances with dict.get(value)),
remove it.