Split process into two steps. First parses and bins into files based on date, t
Split the process into two steps. Preliminary step parses and bins the entries into files by date. Second step calculates frequencies.
One month's worth of files (13GB, 78 million recoreds) now completes on a Macbook Air in under 2 hours.