Major changes to code to handle huge influx of data over multiple days. This version will need some more cleaning up to make it real efficient