porting over some internal code for public release, still a few bugs to go...