Remove memory pool, other critical improvements in performance