Explored the 'raw' data. Couldn't reduce it due to mem issues. Maybe I need to reserve more RAM. Alternatively, I can process each file separately and later join them.