Initial work on some performance improvments, not really a refactor