Attempt to limit how much is ever actually in memory

~~This is a WIP. The tests are failing locally for me and I cannot determine why.~~