Huge clean up to not use any @ managed pointers
This also ended up making my pointer ownership structure quite sane