Wouldn't `defaultdict` work here?