WIP - Some fixes, some speed

Again still WIP, but starting pass to do more testing and speed up some
things, timings...