a couple optimizations to the new method (now about 6x faster than the old method, but no persistence yet)
persistent tables engage (need to cut out the old stuff now)