Refactor how file manifest works

Joe thinks this is abstraction. I'm not so sure